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Operations Research really emerged as an academic discipline with the publication 
in the late 1960s of a series of pioneering introductory textbooks. A great deal 
has happened to the field in the three decades since, but most current introductory 
courses, and most new texts that have emerged along the way, still largely follow the 
treatment pioneered in those seminal books. 

There is much good in that traditional development, but I undertook this new 
text on the deterministic (mathematical programming) half of OR because I believe 
students need something more. Emphasis should be on skills and intuitions they 
can carry away and apply in real settings or later courses. Modeling (or at least 
formulation) should be central. Students should be excited about the versatility and 
applicability of OR tools. 

Most of the innovations of this book try to deliver on that vision. Numerous 
algorithms (including simplex) are developed in the context of a common improving 
search paradigm designed to prepare students for absorbing new material as they 
encounter it within the course and later. The full breadth of mathematical pro- 
gramming (including linear, integer, nonlinear, and multiobjective programming) is 
treated from the beginning to highlight the rich diversity of potential applications. 
Three entire chapters, and large portions of others, are devoted to describing and 
formulating examples, most of which are based on reports of real applications. The 
entire development is structured to facilitate its use in a variety of settings by making 
it as easy as possible to locate material and move around. 

The book is intended for first undergraduate and graduate OR courses in en- 
gineering and mathematics, and undergraduate/graduate electives in management, 
as well as introductory courses in subtopics such as linear programming, integer 
and combinatorial optimization, network flows, and nonlinear programming. Its 
comprehensive and reentrant coverage should also make it a valuable reference for 
practitioners and researchers. 

My rethinking of how to present mathematical programming begins with the 
conviction that you cannot do justice to the topic by teaching only linear pro gramming 
and the simplex method. First of all, the overwhelming majority of optimization 
problems actually encountered in engineering and management practice cannot be 
modeled straightforwardly as linear programs. They involve discrete, nonlinear, 
and multiobjective elements that have to be confronted. Perhaps more importantly, 
OR practice has evolved to the point where integer, nonlinear, and multiobjective 
optimization models are dealt with every day. I was surprised myself, in researching 
the OR applications literature for this book, that accounts of discrete optimization 
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significantly outnumber those using linear programming alone. Goal programs are 
everywhere. If students have never seen anything but classic LP, they will inevitably 
conclude OR is too narrow to deal with the complexities of real planning and design 
tasks. 

Focusing heavily on the simplex algorithm also introduces a more subtle road- 
block to learning. Regardless of whether simplex is taught as a combinatorial search 
for the right basic set or as a sequence of linear equation manipulations, decades of 
refinement on simplex development have almost totally obscured its relation to all 
the other tools of mathematical programming. A student who has learned only the 
simplex now carries away little insight around which to build any understanding of 
the wider field. Even the newer interior point algorithms for linear programming 
seem completely foreign. 

I am not sugggesting that every introductory course must delve deeply into 
integer and nonlinear programming techniques, or that LP and the simplex should be 
discarded. What I believe we need is a development that treats LP as the best solved 
but far from the only form of mathematical program, and that equips students with the 
intuitions and paradigms to understand more advanced topics when they encounter 
them in later courses or practice. Where there is the time or the prerequisites to go 
more deeply into wider topics, that material should be there, developed as much as 
possible around the same concepts as those of basic LP. 

This book tries to avoid equating linear programming with optimization by 
treating other cases from the very beginning. Asin any introduction, the early priority 
is on formulating optimization problems in standard mathematical programming 
format. But by including integer, nonlinear, and multiobjective cases along with LPs, 
students are encouraged to get excited about how vast a range of applications can at 
least be modeled in this standard way. They can also begin early to recognize different 
model classes and to be alert to their relative tractability. My experience is that 
beginners will struggle with intracacies of subscripts, decision variables, constraints, 
and objectives—just as they do if we treat only LPs—but that the broader range of 
mathematical forms is well within their prior training. Binary variables are often the 
only new mathematical concept, and students find them quite intuitive. 

Treatment of the improving search (hillclimbing) paradigm begins in Chapter 
3 with basic notions of improving and feasible directions, step sizes, local and global 
solutions, and artificial starts. With the freedom to use nonlinear examples, this can 
be done in an intuitive and geometic way. Then the simplex algorithm is presented 
in Chapter 5 as an elegant specialization to linear programming. That improving 
search treatment of the simplex will seem unfamiliar even to many instructors, but 
very little is actually changed. Still, with the hillclimbing paradigm firmly established, 
it becomes much easier to develop interior point methods in Chapter 6, network 
algorithms in Chapter 10, nonlinear programming methods in Chapters 13-14, and 
even integer programming procedures in Chapter 12. Time probably will not permit 
covering all those topics in any single course, but my hope is that the paradigm 
prepares students for future learning, whether it comes in the current course, in later 
ones, or in professional practice. 

Students can hardly be expected to get excited about using optimization meth- 
ods if they have never seen them applied except on contrived examples small enough 
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for hand computation. Also formulation skills, which I always find take the longest 
to develop, demand constant practice. These are why I have tried to make the book 
both an exposition of the rich diversity of OR applications and a continuing discourse 
on optimization modeling. Every algorithm and analytic principle is developed in 
the context of a brief story, and many computational exercises begin with a formu- 
lation step. The several whole or partial chapters devoted to formulation are full of 
realistic examples drawn from operations reseach practice, and exercises add many 
others. Time-short instructors will find both long enough to preserve a sense of the 
real application, yet sufficiently self-contained to be presented in 15 to 30 minutes of 
class time. 

All serious mathematical programming computation requires computers, and 
I have tried to reflect that as well in my updated conceptualization of introduc- 
tory OR. One way for students to judge whether their formulations are correct is 
to input them to optimization software and examine the results. That is why ev- 
ery formulation exercise with numerical constants also includes a computer solu- 
tion step. Hand computation is still employed in numerous small exercises to de- 
velop student understanding of principles and algorithms, but here too the computer 
is exploited when possible. A few purely hand topics of traditional introductory 
courses have been dropped altogether because they provide neither tools for solv- 
ing problems of practical size nor insights into the working of effective computer 
algorithms. 

Realizing that few instructors will cover every topic in the book, and that 
even fewer students will read every page, I have tried the make the book as easy 
to move around in as possible. Dependencies between sections are minimized and 
clearly identified with explicit references. The entire development is broken into 
short and carefully labeled subsections addressing one topic each. Every major 
definition, principle and algorithm is set out for quick reference in an easy-to-spot 
box. One- or two-page “Primers” concisely review prerequisite material as it arises 
in the development. Computations and theoretical principles, which may be de- 
rived over several pages, are recapped in brief “Sample Exercises” also marked for 
easy identification. Icons in the exercises indicate which require computers (Q) or 
graphic calculators (&), and which have answers provided at the back of the book 
(2). 

Instructors who consider adopting the book may have to spend some time 
aligning their lectures and other materials with my innovations. Still, the cost is 
much less than it might seem at first glance because the changes have less to do 
with what is taught than the sequence and conceptual framework within which it is 
developed. I have tried to help with sample syllabi, notes, software and other aids 
posted through Internet site http:/www.ecn.purdue.edu/Trardin/oorbook/. 

I hope the effort to convert will prove justified for others. It has certainly 
been rewarded with improved learning and greater student motivation in my own 
classes at Purdue. I owe a great deal of thanks to the hundreds who have aided in its 
development as they studied from emerging drafts. 

L also want to thank my wife Blanca and son Rob for their encouragement and 
patience through the long years this book has been underway, my two supportive 
department heads Ferd Leimkuhler and Marlin Thomas, and a sequence of helpful 
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editors with Macmillan and Prentice-Hall. Special recognition is due editor David 
Johnstone, who launched me on this quest, and was senselessly murdered just as it 
began to take shape. 
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Problem Solving 
with Mathematical Models 


Any student with the most elementary scientific training has encountered the idea of 
solving problems by analyzing mathematical equations that approximate the phys- 
ical realities of the universe we inhabit. Countless questions about objects falling, 
beams shearing, gases diffusing, currents flowing, and so on, are reduced to simple 
computations upon skillful application of one of the natural laws passed to us by 
Newton, Ohm, Einstein, and others. 

The applicable laws may be less enduring, but “operations” problems such 
as planning work shifts for large organizations, choosing investments for available 
funds, or designing facilities for customer service can also be posed in mathematical 
form. A mathematical model is the collection of variables and relationships needed 
to describe pertinent features of such a problem. 


In this chapter some of the fundamental issues and vocabulary are introduced. 


1,] OR APPLICATION STORIES 


Operations research techniques have proved useful in an enormous variety of appli- 
cation settings. One of the goals of this book is to expose students to as broad a sample 
as possible. All examples, many end-of-chapter exercises, several complete sections, 
and three full chapters present and analyze stories based on OR applications. 

Whenever possible, these problems are drawn from reports of real operations 
research practice (identified in footnotes). Of course, they are necessarily reduced 
in size and complexity, and numerical details are almost always made up by the 
author. Other stories illustrate key elements of standard applications but greatly 
oversimplify, to facilitate quick learning. 
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(a) Customer demand by week (average = 55) 
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(c) Beginning inventories by week 
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(d) Lost sales by week 
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A handful of continuing examples are even smaller and more contrived. They 
still have a story, but convenience in illustrating methodological issues takes prece- 
dence over reality of application. 


EXAMPLE 1.1: MORTIMER MIDDLEMAN 


Our first story is of the totally made-up variety. Mortimer Middleman—friends call 
him MM—operates a modest wholesale diamond business. Several times each year 
MM travels to Antwerp, Belgium, to replenish his diamond supply on the interna- 
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tional market. The wholesale price there averages approximately $700 per carat, but 
Antwerp market rules require him to buy at least 100 carats each trip. Mortimer and 
his staff then resell the diamonds to jewelers at a profit of $200 per carat. Each of 
the Antwerp trips requires 1 week, including the time for Mortimer to get ready, and 
costs approximately $2000. 

Customer demand values in Figure 1.1(a) show that business has been good. 
Over the past year, customers have come in to order an average of 55 carats per 
week. 

Part (c) of Figure 1.1 illustrates Mortimer’s problem. Weekly levels of on-hand 
diamond inventory have varied widely, depending on the ups and downs in sales and 
the pattern of MM’s replenishment trips [Figure 1.1(b)]. 

Sometimes Mortimer believes that he is holding too much inventory. The 
hundreds of carats of diamonds on hand during some weeks add to his insurance 
costs and tie up capital that he could otherwise invest. MM has estimated that these 
holding costs total 0.5% of wholesale value per week (i.e., 0.005 x $700 = $3.50 per 
carat per week). 

At other times, diamond sales—and Mortimer’s $200 per carat profit—have 
been lost because customer demand exceeded available stock [see Figure 1.1(d)]. 
When a customer calls, MM must either fill the order on the spot or lose the sale. 

Adding this all up for the past year, MM estimates holding costs of $38,409, 
unrealized profits from lost sales of $31,600, and resupply travel costs of $24,000, 
making the annual total $94,009. Can he do better? 


1.2 OPTIMIZATION AND THE OPERATIONS RESEARCH PROCESS 


Operations research deals with decision problems like that of Mortimer Middleman 
by formulating and analyzing mathematical models—mathematical representations 
of pertinent problem features. Figure 1.2 illustrates this OR process. 

The process begins with formulation or modeling. We define the variables and 
quantify the relationships needed to describe relevant system behavior. 

Next comes analysis. We apply our mathematical skills and technology to see 
what conclusions the model suggests. Notice that these conclusions are drawn from 


assessment 


inference 


FIGURE 1.2 Operations Research Process 
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the model, not from the problem that it is intended to represent. To complete the 
process, we must engage in inference, that is, argue that conclusions drawn from the 
model are meaningful enough to infer decisions for the person or persons with the 
problem. 

Often, an assessment of decisions inferred in this way shows them to be too in- 
adequate or extreme for implementation. Further thought leads to revised modeling, 
and the loop continues. 


Decisions, Constraints, and Objectives 
We always begin modeling by focusing on three dimensions of the problem: 


are (a) the decisions open to decision makers, (b) the constraints limiting deci- 


sion choices, and (c) the objectives making some decisions preferred to others. 

In dealing with virtually any decision problem—engineering, management, or 
even personal—explicitly defining the decisions, constraints, and objectives helps to 
clarify the issues. Mortimer is obviously the decision maker in our diamond inventory 
management example. What decisions does he get to make? 

Actually, MM makes hundreds of decisions each year about when to replenish 
his stock and how much to buy. However, it is common in inventory management 
circumstances such as Mortimer’s to reduce the question to two policy decisions: 
What reorder point level of inventory should trigger a decision to buy new stock, and 
what order quantity should be purchased each time? These two variables constitute 
our decisions. We presume that each time on-hand inventory falls below the reorder 
point, Mortimer will head to Antwerp to buy a standard reorder quantity. 

The next issue is constraints. What restrictions limit MM’s decision choices? 
In this example there aren’t very many. It is only necessary that both decisions be 
nonnegative numbers and that the order quantity conform to the 100-carat minimum 
of the Antwerp market. 

The third element is objectives. What makes one decision better than another? 
In MM’s case the objective is clearly to minimize cost. More precisely, we want to 
minimize the sum of holding, replenishment, and lost-sales costs. 

Summarizing in a verbal model or word description, our goal is to choose a 
nonnegative reorder point and a nonnegative reorder quantity to minimize the sum 
of holding, replenishment, and lost-sales costs subject to the reorder quantity being at 
least 100. 


Optimization and Mathematical Programming 


Verbal models can help organize an analyst’s thinking, but in this book we address 
a higher standard. We deal exclusively with optimization (also called mathematical 
programming). 


1.2 Optimization and the Operations Research Process 


With our Mortimer Middleman example, the decision variables are 


q & reorder quantity purchased at on each replenishment trip 
r 4 reorder point signaling the need for replenishment 


(Here and throughout A means “is defined to be.”) Constraints require only that 
q = 100 
ieee) 
The objective function, 
c(q, r) 4 total cost using a reorder quantity of g and a reorder point r 


remains to be explicitly represented mathematically. We seek to minimize c(q, r) 
over values of g and r satisfying all constraints. 


Constant-Rate Demand Assumption 


How we formulate constraints and objectives in terms of decision variables depends 
on what assumptions we are willing to make about the underlying system. We begin 
with a strong assumption regarding constant-rate demand: Assume that demand 
occurs at a constant rate of 55 carats per week. It is clear in Figure 1.1(a) that the 
demand rate is not exactly constant, but it does average 55 carats per week. Assuming 
that it is 55 carats in every week leads to some relatively simple analysis. 

If the demand rate is constant, the pattern of on-hand inventory implied by a 
particular g and r will take one of the periodic “sawtooth” forms illustrated in Fig- 
ure 1.3. Each time a shipment arrives, inventory will increase by order size q, then 
decline at the rate of 55 carats per week, producing regular cycles. Part (a) shows 
a case where inventory never runs out. A safety stock of (theoretically) untouched 
inventory protects against demand variability we have ignored. At the other extreme 
is part (c). Sales are lost because inventory runs out during the lead time between 
reaching the reorder point r and arrival of a new supply. Part (b) has neither safety 
stock nor lost sales. Stock runs out just as new supply arrives. 


Back of Envelope Analysis 


In cases where there are no lost sales [Figure 1.3(a) and (b)] it is easy to compute the 
length of each sawtooth cycle. ; 


order quantity  q 


demandrate 55 
With lost sales [Figure 1.3(c)], each cycle is extended by a period when MM is out of 
stock that depends on both g and r. 
Clearly, both modeling and analysis would be easier if we could ignore the lost- 
sales case. Can we afford to neglect lost sales? As in so many OR problems, a bit of 
crude “back of envelope” examination of the relevant costs will help us decide. 
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inventory 
4 


eo slope = 55 per week (d) 


! == — > time 
a wee, 
1 week (1) @l55 


(a) With safety stock 
inventory 
r 


slope = 55 per week (d) 


q 
| 
r| 
| 
4 time 
—— —— 


1 week (1) 


q/55 
(b) No safety stock or lost sales 


inventory 


f —— slope = 55 per week (d) 


> time 


coo 
1 week (1 ) 


q/55 
(c) With lost sales 


FIGURE 1.3 Inventories Under Constant-Rate Demand 


1.2 Optimization and the Operations Research Process 


Lost sales may occur under the best of plans because of week-to-week varia- 
tion in demand. Under our constant-rate demand assumption, however, there is no 
variation. Furthermore, MM can afford to add a unit to g and carry it for up to 

cost oflostsale _ $200 

weekly holding cost $3.50 

rather than lose a carat of sales. Since the history in Figure 1.1 shows that inventory 

typically has been held no more than 4 to 6 weeks, it seems safe to make a second 
assumption regarding no lost sales: Assume that lost sales are not allowed. 


= 57.1 weeks 


Constant-Rate Demand Model 


Since customers order a constant-rate 55 carats during the 1 week it takes Mortimer 
to carry out an Antwerp trip, both inventory at order arrival and lost sales can be 
computed by comparing 55 to r. Ifr < 55, we lose (55 — r) carats of sales each cycle, 
something we have decided not to permit. Thus we may deduce the constraint 


r>55 


With r restricted to be at least 55, (r — 55) is the safety stock, and the cycle 
of rising and falling inventory repeats every q/55 weeks. Inventory on hand ranges 
from (r — 55) at the low point of a cycle to (r — 55) + q as a shipment arrives. The 
average will be the midpoint of these values, (r — 55) + q/2. 

We are finally in a position to express all relevant costs. Holding cost per week 
is just the average inventory held times $3.50. Replenishment cost per week is $2000 
divided by the cycle length or time between replenishments. Our first optimization 
model is 
aT 2000 


minimize c= 3.50 G —55)+ 5 + q55 (1.1) 


subject to g>100, r>55 
Feasible and Optimal Solutions 


Remember that our goal is to help Mortimer make decisions. Since the decisions are 
the variables in our model, we would like to characterize good values for decision 
variables g and r. 


14 _| A feasible solution is a choice of values for the decision variables that 


of any other feasible solutions. 


For example, g = 200, r = 90 is feasible in constant-rate demand model (1.1) because 
both constraints are satisfied: 200 > 100 and 90 > 55. 

Here we can go farther and find an optimal solution. To begin, notice that if r 
deviates from demand 55, we incur extra holding cost and that no constraint prevents 
choosing r exactly 55. We conclude that 


(=o) 


will tell MM the perfect moment to start travel preparations. The asterisk (*) or star 
on a variable always denotes its optimal value. 
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Substituting this optimal choice of r of (1.1), the objective function reduces to 


c(q, 7) & 3.50 (3) + 2000 (=) (1.2) 


Elementary calculus will tell us how to finish (differentiate with respect to gq and 
solve for a suitable point where the derivative is zero). To avoid being diverted by 
mathematical details in this introductory chapter, we leave the computation as an 
exercise for the reader. 

The graphic presentation of cost function (1.2) in Figure 1.4 confirms the cal- 
culus result that the minimum average weekly cost occurs at 


2(2000) (55) 
* — 4, | —— 950.7 
bag RES 
Since this value easily satisfies the g > 100 constraint, it is optimal. 


weekly cost 
ry 


2000 r 


is 62380 Gy) +2000 (3) 


<p 


1500 


1000 |- 
500 - 
q* = 250.7 
j order 
100 200 300 400 quantity 


FIGURE 1.4 Optimal MM Order Quantity Under Constant-Rate 
Demand 


To summarize, our assumptions of constant-rate demand and no lost sales have 
led us to advise Mortimer to go to Antwerp whenever inventory drops below r* = 55 
carats and to buy g* = 250.7 carats of new diamonds each trip. Substituting these 
values in the objective function of (1.1), total cost should be about $877.50 per week or 
$45,630 per year—quite an improvement over Mortimer’s real experience of $94,009. 


1.3 SYSTEM BOUNDARIES, SENSITIVITY ANALYSIS, 
TRACTABILITY, AND VALIDITY 


The modeling in Section 1.2 took as given many quantities, such as the demand per 
week and the cost per carat held, then computed optimal values for reorder point 


1.3 System Boundaries, Sensitivity Analysis, Tractability, and Validity 


and reorder quantity. A line between those items taken as settled and those to 
be decided is called the system boundary. Figure 1.5 illustrates how parameters— 
quantities taken as given—define objective functions and constraints applicable to 
the decision model inside. Together, parameters and decision variables determine 
results measured as output variables. 


modeling 


and output 
parameters analysis variables 
in the 
decision 
variables 


FIGURE 1.5 System Boundaries 


EOQ Under Constant-Rate Demand 
Only cost c is an output variable in our constant-rate demand model of Mortimer 
Middleman’s problem. Enumerating the parameters, let 

d 4 weekly demand (55 carats) 

f A fixed cost of replenishment ($2000) 

h & cost per carat per week for holding inventory ($3.50) 

s 4 cost per carat of lost sales ($200) 


¢ 4 lead time between reaching the reorder point and receiving a new supply 
(1 week) 


m & minimum order size (100 carats) 


A great attraction of our constant-rate demand analysis is that it can be done just as 
well in terms of these symbols. If lost sales are not allowed, repetition of the analysis 
(calculus) in terms of symbolic parameters will cause us to conclude that 


These results hold as long as g* > m (to be general, we also need to specify 
g > td). 

The square root expression for g* now exhibits one of the oldest results in op- 
erations research: the classic economic order quantity (EOQ) formula for inventory 
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management. Although we will soon see problems with the r* part of the solution in 
MM’s case, EOQ order quantities yield reliable inventory policies in a wide variety 
of settings. 


System Boundaries and Sensitivity Analysis 


To see the power of symbolic results such as equations AS). we must recognize the 


inherent arbitrariness in system boundaries. If we took nothing as settled, models 
would mushroom in complexity and meaningful analysis would become impossible. 
Still, parameters we choose to regard as fixed at a system boundary often are known 
only vaguely. For example, MM’s trips may cost approximately f = $2000, but prices 
no doubt change with time, and more careful tabulation of past expenses might 
show that trips actually average $1000 or $3000. Figure 1.6 shows the significant 
implications for the optimal g*. Our analysis with parameter f = $2000 had q* = 
250.7. With f = $1000, the optimal order quantity is 177.3; and with f = $3000, it is 
307.1. 


weekly cost 
A 


2000 + 
1500 - 
1000 - 
500 F 
{: order 
100 200 300 400 quantity 


FIGURE 1.6 Sensitivity of MM Constant-Demand Results to 
Parameters 


Such variations in input parameter values taken as fixed at a system boundary 
may dramatically affect the results of OR analysis. 


sloration of results from mathematical mode 
é values oer for Palapetets 


Any really complete operations research study includes an investigation of the sen- 
sitivity of results to parameter values. 


1.4 Descriptive Models and Simulation 


Closed-Form Solutions 
Solutions prescribing choices for decision variables by simple formulas in the input 


variables are called closed-form solutions. Expressions | 1.5 | are an example. 

The power of closed-form solutions lies with their providing results for many 
values of the parameters. Determining optimal g* and r* for the parameter values 
assumed in Section 1.2 was no small success. With closed-form solutions, however, 
we can determine sensitivities of optimal results to changes in input parameters. 
For example, we can see immediately that travel cost parameter f influences g* in 
proportion to its square root and has no impact on 7*. 


Our constant-rate demand model of Mortimer Middleman’s problem has proved 
highly tractable because elementary calculus produced closed-form optimal solu- 
tions. Should we conclude that the goal of every OR study must be to formulate and 
analyze a model admitting closed-form solutions? Absolutely not! 

Take another look at the operations research process depicted in Figure 1.2. 
Our purpose is not mathematical elegance but insights that will really help people 
like Mortimer Middleman deal with their problems. To assess the merit of operations 
research results, we must also consider another dimension. 


The complete analysis possible with our constant-rate demand model derived 
directly from the strong assumptions that we made to achieve a simple mathematical 
form. As we will see in the following subsections, those same assumptions bring the 
applicability of our closed-form results into real doubt. 

Our study of OR will return to this dilemma again and again. 


1.10 | ORanalystsa 


and their tractabilits 


1.4 DESCRIPTIVE MODELS AND SIMULATION 


Our constant-rate demand analysis of Mortimer Middleman’s problem started by 
reducing all the information in Figure 1.1(a) to a single number, an average of 55 
carats sold per week. Why not use more of what we know? 
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Simulation over MM’s History 


A simulation model is a computer program that simply steps through the behavior 
of a system of interest and reports experience. Of course, the behavior is tracked in 
computer variables and program logic rather than in a physical system. 

To illustrate for Mortimer Middleman’s inventory problem, suppose that we 
retain our verbal model (i.e., continue to focus on a reorder point r and a reorder 
quantity qg). A straightforward computer program could then step through the 52 
weeks of available history to try any given r and qg. At each step, the program would: 


J. Check whether MM is due to arrive with a new order of size q. 
2. Determine if r has been reached so that a new trip is needed, 
3. Reduce inventory by the demand actually experienced. 


Holding cost for the week can be reported as $3.50 times the average number 
of carats in inventory. Replenishment cost is $2000 in each trip week. Lost-sales cost 
for the week is $200 times any excess of demand over the available inventory. 

Table 1.1 details such a simulation from MM’s demand history. Using the op- 
timal g* = 251 and r* = 55 computed in our constant-rate demand model, this 
simulation reports beginning inventory for each week, customer demand taken from 
Figure 1.1(a), inventory management actions taken, and their consequences for hold- 
ing cost, replenishment cost, and lost sales. 

Total simulated cost for all 52 weeks is $108,621. Recall that Mortimer esti- 
mated actual cost at $94,009, and the constant demand analysis said that costs under 
q* and r* should decline to $45,630. The simulation model now is telling us that 
if MM adopts our supposedly optimal constant-rate demand policy, he will spend 
$108, 621 — $94, 009 = $14, 621 more than if he simply keeps operating as he always 
has. This is hardly a help. 


Simulation Model Validity 


Can these new results be trusted? Only a clairvoyant could be certain, but it seems 
safe to conclude that the simulation results, based on an entire year’s actual expe- 
rience, should be taken more seriously than values derived from the constant-rate 
demand model. 


i 11_| Simulationm dels often possess high vay because they. track | 
tem behavior fairly | 


It should be noted, however, that the simulation was also based on some as- 
sumptions. For example, we have implicitly presumed that future demand will exactly 
mirror last year’s experience. 


Descriptive versus Prescriptive Models 


Think now about tractability. How much did our simulation actually tell us? It 
estimated the holding, replenishment, and lost-sales costs of operating with g = 251 
and r = 55 using fixed values for input parameters. Nothing more. 

Models that evaluate fixed decision alternatives rather than indicating good 
choices may be termed descriptive models. Evaluating a few specific choices in this 


1.4 Descriptive Models and Simulation 


TABLE 1.1 Deterministic Simulation of MM's Problem from Prior History 


Week, 


t 


* aa 
OAADMBHWNFrFTCWAAIADUMABWNH 


way sometimes tells a decision maker (here MM) as much as he or she needs to know. 


Inventory 


100 


Beginning Customer 


Demand 


Simulated 
Action 


Sell 94 

Below r=55, so trip; sell 6 
q = 251 arrive; sell 52 

Sell 64 

Sell 69 

Sell 66 

Below r=55, so trip; sell 0 
q =251 arrive; sell 47 

Sell 68 

Sell 56 

Sell 62 

Below r=S5, so trip; sell 18 
q = 251 arrive; sell 41 

Sell 46 

Sell 84 

Sell 80 

Below r=SS, so trip; sell 0 
q =251 arrive; sell 52 

Sell 67 

Sell 26 

Sell 59 

Below r=SS, so trip; sell 47 
q =251 arrive; sell 42 

Sell 59 

Sell 11 

Sell 67 

Sell 25 

Below r=55, so trip; sell 47 
gq = 251 arrive; sell 41 

Sell 42 

Sell 47 

Sell 66 

Below r=55, so trip; sell 20 
q = 251 arrive; sell 46 

Sell 36 

Sell 69 

Sell 64 

Sell 71 

Below r=SS, so trip; sell 0 
q=251 arrive; sell 38 

Sell 13 

Sell 50 

Sell 77 

Sell 64 

Below r=55, so trip; sell 9 
q =251 arrive; sell 96 

Sell 57 

Sell 95 

Below r=55, so trip; sell 3 
q = 251 arrive; sell 56 

Sell 68 

Sell 42 


Holding Replenishment 


Cost Cost 
$185.5 0 
ie $2,000 
787.5 0 
584.5 0 
353.5 0 
110.5 0 
0.0 2,000 
798.0 0 
595.0 0 
378.0 0 
171.5 0) 
12.9 2,000 
808.5 0 
654.5 0 
427.0 0 
119.1 0 
0.0 2,000 
787.5 0 
581.0 0 
416.5 0 
269.5 0 
50.2 2,000 
805.0 0 
630.0 0 
507.5 0 
371.0 0 
210.0 0 
64.4 2,000 
808.5 0 
661.5 0 
507.5 0 
308.0 0 
1575 2,000 
920.5 0 
777.0 0 
595.0 0 
360.5 0 
106.3 0 
0.0 2,000 
812.0 0 
724.5 0 
612.5 0 
392.0 0 
143.5 0 
2 2,000 
710.5 0 
444.5 0 
178.5 0 
0.3 2,000 
780.5 0 
563.5 0) 
371.0 0 


After all, many problems admit only a few practical solutions. 


Still, results for a few cases are a very long way from what our prescriptive 
constant-rate demand optimization provided. Section 1.3’s closed-form results both 


Lost 
Sales 


0 
$ 9,600 
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2,60 
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recommended optimal choices for g and r and offered insight about the sensitivity 
of results to changes in parameter values. The simulation did neither. 


Id fewer analytic inferences than prescriptive, op- 
take both input parameters and de 


1.12 | Descriptive models y 
- timization models becau: 
a 


The tradeoff should now be clear. Resorting to a mathematically structureless 
form such as a simulation model can substantially improve validity. Still, that same 
lack of mathematical structure severely limits the possible analysis. More model 
validity almost always implies less tractability. 


1.5 NUMERICAL SEARCH AND EXACT VERSUS HEURISTIC 
SOLUTIONS 


The simulation in Section 1.4 provided a computer program for estimating the to- 
tal inventory cost associated with a particular choice of reorder point and reorder 
quantity. Suppose we think of that computation as a function c(q, r). That is, for any 
given g andr, 


c(q, r) 4 total cost computed by the simulation with reorder point fixed 
at r and reorder quantity at q 


Mortimer Middleman’s problem then reduces to the mathematical model 
minimize  c(q, r) 
subject to g>100, r>0 


(1.3) 


Numerical Search 


Since we know very little about the properties of the mathematical function c(q, r), 
restating our problem in this abstract form lends no immediate insight. Still, it does 
suggest a way to proceed. 

Numerical search is the process of systematically trying different choices for the 
decision variables, keeping track of the feasible one with the best objective function 
value found so far. We call such a search numerical because it deals with specific 
values of the variables rather than with symbolic quantities such as those we were 
able to manipulate in analyzing the constant-rate demand model. 

Here we search numerically over g and r. It seems reasonable to begin with 
the q and r recommended by our analysis of the constant-rate demand model. Using 
superscripts (note that these are not exponents) to identify specific choices of the 
decision variables, g = 251, r = 55, and we have already seen that c(q, rf) = 
$108, 621. 

Next, we need a systematic process for thinking of new q’s and r’s to try. Much 
of this book centers on how search processes should be structured. For now, we will 
try something naively simple: increasing and decreasing one variable at a time in 
steps of 10. 

Table 1.1 showed considerable lost sales, so we start by increasing r to introduce 
a safety stock. Continuing until the objective function deteriorates yields. 


1.5 Numerical Search and Exact versus Heuristic Solutions 


g® =251 pO = 55 c(q, r©) = 108,621 
qo =o T= 65 c(q®, r) = 108,421 
go = 25) r2) = 75 c(q®, r®) = 63,254 
gq?) = 251 r9) = 85 c(q®, r®) = 63,054 
gv = 05 r= 95 c(q®, r) = 64,242 
All these (q®, r®) are feasible. Proceeding from one of the best, r = 85, we 
now try changing q. Increasing gives 
GPs 26h) j par 8Sin uee(q? 095,193 
and decreasing yields 
Qo = 2A or? 85 ecg”) ro y= 12, 181 


Both are worse than (q®, r®), so we terminate the search. 


A Different Start 


Our numerical search has discovered a choice of g and r with simulated cost $63,054, 
far better than either the $108,621 of the constant-rate demand solution or MM’s 
$94,009 actual cost. Lacking any information on how much MM’s costs might be 
reduced, our only way of learning more with numerical search is to try a new search 
from a different initial point. Consider the search sequence 


gq =251  r =145 (qr) = 56,904 
gq = 251 r) = 155 c(q®, r®) = 59,539 
gq? = 251 r® = 135 c(q™, r™) = 56,900 
gq? =251 r®) = 125 e(q®, 9) = 59,732 
q® = 261 r& = 135 c(q®, r®) = 54,193 
g® =271 r©) = 135 c(q®), r©) = 58,467 


This time, we have happened upon the better heuristic solution g = 261, r = 135 
with simulated cost $54,193. Certainly, the earlier best of g = 251, r = 85 was not 
optimal, but we still have no real reason to believe that our last result is the best 
achievable. 

With only two variables we might try a more exhaustive search, forming a 
loop and evaluating c(q, r) at an entire grid of points. The essential difficulty would 
remain, because results would depend on the size of the grid investigated. 


isl ss IMEreriees from ee search are limited to the a points ox. 


toan n optimization model _ 
function. vane & heuristic or 
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approximate optimum is isa a feasible solution derived from pr cr 


Our numerical searches of MM’s problem have produced only a heuristic optimum— 
a good feasible solution. Should we demand an exact optimum? 

As usual, the answer is far from clear. The three “optimal” values of g depicted 
in Figure 1.6 are all mathematically exact, yet the decisions they recommend vary 
dramatically with the assumed value of input parameters. Furthermore, our simu- 
lation results have shown the true cost to be very different from any of the optimal 
objective function values computed assuming constant-rate demand. 


Still, it would clearly make us more certain of whether to recommend the 
q = 261, r = 135 solution, which is the best uncovered in our numerical searches, 
if we knew something about how far from optimal it might be. Exact optima add a 
satisfying degree of certainty. 


= = model assumptions. 


1.6 DETERMINISTIC VERSUS STOCHASTIC MODELS 


All the searching in Section 1.5 was based on the assumption that Mortimer Mid- 
dleman’s future week-to-week demand will repeat identically the experience of the 
past year. This is almost certainly untrue. The best we can honestly claim regarding 
future events is that we know something about the probability of various outcomes. 


~ quantities known only in probability. 


Random Variables and Realizations 


Random variables represent quantities known only in terms of a probability in 
stochastic models. We distinguish random variables from ones with single values 
by using uppercase (capital) letters. 

If we do not accept the notion that next year’s demands in Mortimer’s problem 
will exactly duplicate last year’s, weekly demands are random variable parameters 
to his decision problem. Each weekly demand will eventually be a specific number. 
At the time we have to choose a reorder point and reorder quantity; however, we 
may know only something about their probability distributions. 


1.6 Deterministic versus Stochastic Models 


If the magnitude of each week’s random demand—denote it D,—is indepen- 
dent of all others (an assumption), each of the values in Figure 1.1(a) is a realization 
of each D, (i.e., a specific historical outcome). We can learn about the probability 
distribution of the D; by counting how often different realizations appear in Fig- 
ure 1.1(a). The solid line in Figure 1.7 presents such a frequency histogram. De- 
mands ranged from 11 to 96, but Figure 1.7 clearly shows that demands in the range 
40 to 70 are most common. 


frequency, probability 


density 
15\— 
10 
5-01 
| weekly 


10 30 50 70 90 demand 


FIGURE 1.7 Weekly MM Demand Frequency 
Distribution 


Eliminating some of the raggedness, the dashed lines in Figure 1.7 show how 
the true probability distribution for D probably looks. Outcomes in the range 40 to 
70 are three times as probable as those in the regions 10 to 30 and 70 to 100, but 
demands are equally likely within regions. 


Stochastic Simulation 


The simulation model of Sections 1.4 and 1.5 is deterministic because all input param- 
eters, including the week-by-week demands, are assumed known when computations 
such as Table 1.1 begin. We can develop a stochastic model of Mortimer’s problem by 
improving on that simulation. Assuming that weekly demands D;, ..., Dsz are inde- 
pendent random variables with known probability distribution such as the smoothed 
one (dashed lines) in Figure 1.7, we will investigate the distribution of the annual 
cost [now random variable C(q, r)] associated with any choice of reorder quantity g 
and reorder point r. 

Stochastic simulation (sometimes called Monte Carlo analysis) provides the 
tool. It samples realizations from output variable distributions by: 


1. Randomly generating a sequence of realizations for input parameters. 
2. Simulating each realization against chosen values for the decision variables. 


With a large enough (and random enough) sample, a frequency histogram of output 
realizations will approximate the true output distribution. 

Figure 1.8 shows results of such stochastic simulation sampling in the Mortimer 
Middleman problem. A total of 200 different sets of realizations for demand variables 
D,,..., Ds2 were randomly sampled from the distribution depicted in Figure 1.7. 
Then each was simulated with gq = 261, r = 135 just as in Table 1.1, and the annual 
cost computed as one realization of C(261, 135). 


sk, 


18 Chapter One e Problem Solving with Mathematical Models 


frequency 
A 
40 4 
q = 261, r = 135 
average cost = $57,374 
range = $52,445 to $69,539 
30 - below $55,000 = 31 of 200 


20 5 


10 4 


mn . 
50 55 60 65 70 75 cost ($000) 


FIGURE 1.8 MM Annual Cost Frequency Distribution 


We can see from Figure 1.8 that for this best known choice of decision variables, 
annual cost has a distribution ranging from $52,445 to $69,539, with an average of 
about $57,374. Notice that this range of possible futures does include the single value 
$54,193 obtained in Section 1.5’s numerical searches. Still, many other costs might 
result, depending on what demand pattern is actually realized. 

Stochastic simulation is an effective and widely used OR modeling technique 
because the enormous flexibility of simulation models makes it possible to formu- 
late validly a very wide range of problem situations. Still, the ragged shape of the 
frequency histograms in Figure 1.8 highlights the extra analytic challenge associated 
with computing distributions by simulation. Even with decision variables fixed, and 
after 200 runs, we know the C(q, r) distributions in Figure 1.8 only approximately. 
Statistical estimation techniques would be required to determine how much confi- 
dence we can have in any conclusions. 


Tradeoffs between Deterministic and Stochastic Models 


Few, if any, parameters of real systems can actually be quantified with certainty, yet 
that is exactly what deterministic models assume. Distributional estimates such as 
Figure 1.8 usually provide a much more complete picture of what might actually 
happen with any chosen q and r. 

Why not use stochastic models for every problem? It should be no surprise 
that the answer is: tractability. We have already seen how much work is required to 
estimate the output distribution for a single choice of g andr via stochastic simulation. 
Other stochastic methods can sometimes produce results with less effort, and validity 


1.7. Perspectives 


sometimes demands that stochastic variation be modeled, but a major tractability 
difference remains. 


When optimization is the goal, deterministic modeling is often the only practical 
choice. 


1.20. | Most optimization models are -deterministi 
really believe that all problem parameters are 
useful fescuptive a can often be o y 
isignored. — ] 


ot because OR analysts 
ith certainty, but because 


i tochastic variation. 


Of course, deterministic optimization models are also used because they work. 
We will see in illustrations throughout this book that very satisfactory results have 
been obtained in a wide variety of applications, often in circumstances that involve 
many thousands of decision variables. 


|,'7 PERSPECTIVES 


Mortimer Middleman’s inventory problem is only one of hundreds of operations 
design, planning, and control questions that we will encounter in this book. The 
purpose of treating it in so many different ways here is to introduce the possible 
approaches and to reveal some of their strengths and weaknesses. 

OR analysts must daily decide whether a back-of-the-envelope computation is 
sufficient or a formal model is required; whether a more detailed and thus more valid 
model is preferred to a more tractable approximation. One goal of this book is to 
provide the vocabulary and sensitivity to issues that will help us make those choices. 


Other Issues 


The chapter’s emphasis on the tractability versus validity tradeoff in operations re- 
search is not intended to suggest that it is the only issue. Models also differ in how eas- 
ily they can be understood by the consumer of the analysis (Mortimer here), how fast 
analysis must be completed, how much mathematical and computer power is required 
to do the analysis, how many data must be collected, and in a variety of other ways. 

More fundamentally, many situations do not lend themselves to the OR ap- 
proach. Even from this chapter’s brief introduction, it should be clear that it is not 
easy to formulate valid models that are tractable to analysis. Operations research is 
founded on the conviction, buttressed by a long history of successful practice, that 
formulation and analysis of mathematical decision models is often worth the trouble. 
Still, time and resources are required. 


1.21 | The model-based OR approach to probl i 
lems important enouen to wart 1 


solving _ works best on prob- 
fora careful study. 
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The Rest of This Book 


In the remainder of this book, the main classes of deterministic optimization models, 
and the solution techniques available to deal with them, are developed. Chapter 
2 begins with formulation and classification of a series of real-world problems in 
terms of the mathematical properties of the models required. Later chapters address 
each class in turn, emphasizing formulation of important business and engineering 
applications and highlighting common elements of the sometimes bewildering variety 
of analysis techniques used to address various classes. 


EXERCISES 


1-1 A segment of automatically controlled highway 
is being equipped with sensors spaced equally along 
its length. The maximum error in estimating traf- 
fic volume, which occurs halfway between any two 
sensors, can be expressed as (d/s)?, where d is the 
length of the segment and s is the number of sensors. 
Each sensor costs p dollars, and designers want to re- 
duce the maximum error as much as possible within 
a budget for sensors of b dollars. Identify each of the 
following for this design problem. 


(a) [] The decision variable 
(b) [Q] The input parameters 
(c) [9] The objective function 
(d) [&] The constraints 


1-2 Return to the problem of Exercise 1-1 and as- 
sume that d = 10, p = 3.5, and b = 14. Determine 
(if necessary by trying all the possibilties) whether 
each of the following is a feasible and/or optimal 
solution. 


(a) Kls=4 
(b) Rls=6 
(c) Qs=2 


1-3 A factory has two production lines available to 
make a product. The first can produce one lot of 
the product in ¢, hours at cost c;, and the second 
requires ft) hours and cost c2. The plant manager 
wishes to find the least costly way to produce b lots 
in a total of at most T hours. An integer number x, 
will be produced on line 1, and integer number x2 on 
line 2. Identify each of the following for this design 
problem. 


(a) The decision variables 
(b) The input parameters 
(c) The objective function 
(d) The constraints 


1-4 Return to the problem of Exercise 1-3 and as- 
sume that t; = 10, H = 20,c; = 500, cp = 100, b = 3, 
and T = 40. Determine (if necessary by trying all 
the possibilties) whether each of the following is a 
feasible and/or optimal solution. 


(a) x, =0,x% =3 
(b) Kp — Dio Ho 
(c) x = 3x10 


1-5 A university wishes to purchase the maximum 
number of computer workstations that can be ac- 
commodated within the available laboratory floor 
space and budget. Determine whether each of the 
following outcomes is most likely the result of closed- 
form optimization, exact numerical optimization, 
heuristic optimization, or descriptive modeling. 


(a) {| The maximum number of stations that can 
be fit within 2000 square feet and a budget of 
$500,000 is 110. 

(b) The usual arrangement of 80 workstations 
would require 1600 square feet of floor space 
and cost $382,000. 

(c) [| The maximum number of stations feasible 
with an area of f thousand square feet and a 
budget of b thousand dollars is min{50f b/5}. 

(d) The best of the usual layouts for an area of 2000 
square feet and a budget of $500,000 can accom- 
modate 85 workstations. 


1-6 Explain why the first alternative of each pair 
constitutes more complete analysis of an optimiza- 
tion model than the second. 


(a) Closed—form versus numerical optimization 

(b) Exact optimization versus heuristic optimiza- 
tion 

(c) Heuristic optimization versus simulation of 
some specific solutions 


1-7 Explain why a model that admitted one of the 
preferred choices in Exercise 1-6 would not neces- 
sarily be more appropriate than one allowing only 
the alternative. 

1-8 An engineer is working on a design problem with 
n parameters to be chosen, and each has 3 possible 
values. A highly valid descriptive model is available, 
and he is thinking of choosing the best design simply 
by using the model to evaluate the effect of every 
combination of design parameter values. 


(a) Explain why this approach will require running 
the descriptive model on 3” combinations of 
decision-variable values. 

(b) [| Forn = 10, 15, 20, and 30, compute the time 
a computer would require to check all the com- 
binations, assuming that it runs 24 hours per day, 
365 days per year, and that it requires 1 second 
to apply the model for any particular choice of 
design parameters. 

(c) Comment on the practical limits of this “try all 
combinations” analysis strategy in the light of 
your results from part (b). 
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1-9 Determine whether each of the following could 
probably be validly modeled only as a random vari- 
able or if a deterministic quantity would suffice. 


(a) [| The number of inches of rainfall in a city 
over the next 14 days 

(b) The average 14-day rainfall in a city 

(c) [| The market price of acommon stock 1 week 
ago 

(d) The market price of a common stock 1 week 
from today 

(c) &] The seating capacity of a restaurant 

(f) The number of customers who will arrive at a 
restaurant this evening 

(g) [| The production rate of an industrial robot 
subject to frequent breakdowns 

(h) The production rate of a highly reliable indus- 
trial robot 

(i) [&] Factory demand for a model of bulldozer 
over the next 7 days 

(j) Factory demand for a model of bulldozer over 
the next 7 years 


i ee eee a 


Deterministic 
Optimization Models 
in Operations Research 


With this chapter we begin our detailed study of deterministic models in operations 
research—models where it is reasonable to assume all problem data to be known with 
certainty. Few who have ever worked with real models can say the words known with 
certainty without breaking a smile. Input constants in OR models are almost always 
estimated, some rather crudely. We employ deterministic models because they often 
produce valid enough results to be useful and because deterministic models are 
almost always easier to analyze than are their stochastic counterparts. 

The increased tractability of deterministic models permits us the luxury of deal- 
ing explicitly with optimization. Often, we achieve only an approximation, but the 
consistent goal is to prescribe the best possible solution. 

Deterministic optimization models are also called mathematical programs be- 
cause they decide how to plan or “program” activities. Our treatment is introduced 
in this chapter with a stream of examples. By formulating models for a variety of 
cases, we begin developing the modeling skills essential to every OR analyst. At the 
same time we illustrate the wide range of model forms available and introduce some 
terminology. Except for a tiny first example, all models presented are based on real 
applications by real organizations. 


>. DECISION VARIABLES, CONSTRAINTS, AND OBJECTIVE 
FUNCTIONS 


From the very early days of deterministic optimization, one of its heaviest users has 
been the petroleum refining industry. Refining operations are routinely planned by 
formal optimization, often on a daily or even hourly basis. We begin our survey 
of mathematical programming models with a made-up example from the refining 
setting, obviously much oversimplified. 


Z3 
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EXAMPLE 2.1: TWO CRUDE PETROLEUM 


Two Crude Petroleum runs a small refinery on the Texas coast. The refinery distills 
crude petroleum from two sources, Saudi Arabia and Venezuela, into three main 
products: gasoline, jet fuel, and lubricants. 

The two crudes differ in chemical composition and thus yield different product 
mixes. Each barrel of Saudi crude yields 0.3 barrel of gasoline, 0.4 barrel of jet fuel, 
and 0.2 barrel of lubricants. On the other hand, each barrel of Venezuelan crude 
yields 0.4 barrel of gasoline but only 0.2 barrel of jet fuel and 0.3 barrel of lubricants. 
The remaining 10% of each barrel is lost to refining. 

The crudes also differ in cost and availability. Two Crude can purchase up to 
9000 barrels per day from Saudi Arabia at $20 per barrel. Up to 6000 barrels per 
day of Venezuelan petroleum are also available at the lower cost of $15 per barrel 
because of the shorter transportation distance. 

Two Crude’s contracts with independent distributors require it to produce 2000 
barrels per day of gasoline, 1500 barrels per day of jet fuel, and 500 barrels per day 
of lubricants. How can these requirements be fulfilled most efficiently? 


Decision Variables 


The first step in formulating any optimization model is to identify the decision 
variables. 


Numerous quantities are floating around in even the very simple Two Crude 
problem statement. Which do we get to decide? Cost, availabilities, yields, and 
requirements are all input parameters—quantities that we will take as fixed (see 
Section 1.3). What must be decided is how much of each crude to refine. Thus we 
define decision variables 

x; 4 barrels of Saudi crude refined per day (in thousands) 


(2.1) 


x2 4 barrels of Venezuelan crude refined per day (in thousands) 


Notice that like good modelers in all fields, we have specified both the meaning of 
each variable and the units in which it is denominated. 


Variable-Type Constraints 
The next issue in formulating an optimization model is constraints. What limits 
decisions? 

The most elementary constraints declare variable type. 


For example, variables may be limited to nonnegative values or to nonnegative 
integer values, or they may be totally unrestricted. 
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Decision variables x; and x2 in the Two Crude example represent quantities 
of petroleum refined. Thus they are subject to the most common variable-type con- 
straint form: nonnegativity. Every meaningful choice of these decision variables 
must satisfy 


Mi eee (2.2) 


That is, both quantities must be nonnegative real numbers. 

It may seem a bit fastidious to specify constraints (2.2) when they are already 
implicit in definitions (2.1). Still, nothing can be taken for granted because our goal is 
to produce a form suitable for analysis by computer-based procedures. Formal meth- 
ods for solving mathematical programs enforce only constraints explicitly expressed 
in the model formulation. 


Main Constraints 


The remainder of the limits on decision variable values constitute the main con- 
straints. 


teractions, other than variable type, that limit decision variable values. 


Even in as simple a case as Two Crude Petroleum, we have several main con- 
straints. First consider output requirements. Petroleum volumes selected must meet 
contract requirements in the sense that 


‘a (yield per barrel) (barrels purchased) > product requirements 
Se ——eEeEeEeEE 


total output of a product 


Quantifying in terms of our decision variables produces one such constraint per 
product: 


0.3x1 + 0.4x2 > 2.0 (gasoline) 
0.4x1 +0.2x2 > 1.5 (jet fuel) (2.3) 
0.2x; + 0.3x2 = 0.5 (lubricants) 


Each is denominated in thousands of barrels per day. 

Availabilities yield the other class of main constraints. We can buy no more than 

9000 barrels of Saudi crude per day, nor 6000 barrels of Venezuelan. Corresponding 
constraints are 

x, <9 (Saudi) . 


X2 <6 (Venezuelan) (2.4) 


Objective Functions 


Objective or criterion functions tell us how to rate decisions. 
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What makes one choice of decision variable values preferable to another in the 
Two Crude case? Cost. The best solution will seek to minimize total cost: 


min ~ (crude unit cost) (barrels purchased) 
Quantifying this single objective in terms of the decision variables yields 
min 20x; + 15x (2.5) 
(in thousands of dollars per day). 


Standard Model 


Once decision variables have been defined, constraints detailed, and objectives quan- 
tified, the mathematical programming model we require is complete. Still, it is cus- 
tomary to follow variable and parameter definitions with a summary of the model in 
a standard format. 


The standard statement of an optimization model has = ese 


min ormax (objective function(s)) 
st. __—~—_ (main constraints) 
(variable-type oe 


where “s.t.” stands for “subject 1 fo 


Combining (2.2)-(2.5), we may formalize our Two Crude model as follows: 
min 20x; + 15x. (total cost) 
st. 0.3x1 + 0.4x2 > 2.0 (gasoline requirement) 


0.4x; + 0.2x2 > 1.5 (jet fuel requirement) 


0.2x1 + 0.3x2 > 0.5 (lubricant requirement) (2.6) 
x1 ©) (Saudi availability) 

X2 <6 (Venezuelan availability) 

X1,X2 > 0 (nonnegativity) 


SAMPLE EXERCISE 21: FORMULATING FORMAL OPTIMIZATION MODELS | 


Suppose that we wish to enclose a rectangular equipment yard by at most 80 me- 
ters of fencing. Formulate an optimization model to find the design of maximum 
area. 


Modeling: The decisions required are the dimensions of the rectangle. Thus define 
decision variables: 


é 4 length of the equipment yard (in meters) 


w A width of the equipment yard (in meters) 


2.2 Graphic Solution and Optimization Outcomes 


Both variables are nonnegative quantities, which implies variable-type constraints 
l,w>0 


The only main constraint is that the perimeter of the equipment yard should not 
exceed 80 meters in length. In terms of decision variables, that limitation can be 
expressed as 


2£ + 2w < 80 


Our objective is to maximize the enclosed area £w. Thus a complete model 
statement is as follows: 


max fw (enclosed area) 
st. 2€--2w =< 80 (fence length) 
Low >0 (nonnegativity) 


2,2, GRAPHIC SOLUTION AND OPTIMIZATION OUTCOMES 


Methods for analyzing optimization models are the focus of many chapters to fol- 
low. However, very simple graphic techniques have enough power to deal with tiny 
models such as Two Crude formulation (2.6). They also provide “pictures” yielding 
helpful intuition about properties and solution methods for models of more realistic 
size. 

In this section we develop the techniques of graphic solution. We also illustrate 
their intuitive power by exploring the unique optimal solution, alternative optimal 
solution, and infeasible and unbounded outcomes of optimization analysis. 


Graphic Solution 


Graphic solution solves 2 and 3-variable optimization models by plotting elements 
of the model in a coordinate system corresponding to the decision variables. For 
example, Two Crude model (2.6) involves decision variables x; and x2. Every 
choice of values for these variables corresponds to a point (x1, x2) in a 2-dimensional 
plot. 


Feasible Sets 


The first issue in graphic solution is the feasible set (also called the feasible region or 
the feasible space). 


the collection of 


Graphing Constraints and Feasible Sets 


We want to draw a picture of the feasible set in a coordinate system defined by 
the decision variables. The process begins with the variable-type constraints (defin- 


ition ). 
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In the Two Crude example, both variables are nonnegative. Thus every feasible 
solution corresponds to a point in the shaded, nonnegative part of the following 
plot: 


no WwW FF HH DN CO © 


i 


Next we introduce the main constraints (definition | 2.3}) one at a time. Begin 


with the Two Crude gasoline requirement 

0.3x, + 0.4x2 > 2 (2.7) 
If it were of the equality form 

0.3x1 + 0.4x2 = 2 


feasible points would lie along a corresponding line. 


To identify feasible points, we first plot the boundary and then include whichever 
side of the line applies. 


2.2 Graphic Solution and Optimization Outcomes 


Over Two Crude’s nonnegative (x1, x2), the result is 


x2 


gt 


Points along the highlighted boundary satisfy the constraint with equality. To deter- 
mine which side of the line depicts points satisfying the constraint as a strict inequal- 
ity, we have only to substitute one point not on the line. For example, the origin 
xX; =X. = 0 violates the constraint because 


0.3x; + 0.4x2 = 0.3(0) + 0.4(0) # 2 
Feasible points must be on the other side of the equality line, just as the arrow on 
the constraint indicates. 


Feasible (x1,x2) must satisfy all constraints simultaneously. To identify the 
feasible set fully, we merely repeat the foregoing process. 


(or region) fo an optimization n r 
troducing constraints one by one, ping track of the region esha all at 
the ame time. rr Ze : 


Figure 2.1 shows the result for our Two Crude example. Each of the 5 main 
inequality constraints yields a line, where it is satisfied as an equality, plus a side 
of the line where it holds as an inequality. Together with variable-type constraints 
X},% > 0, these define the shaded feasible set. 


SAMPLE EXERCISE 2.23 GRAPHING CONSTRAINTS AND FE IBLE SETS: : 


Graph the feasible sets corresponding to each of the following systems of con- 
straints. 
(a) x1 +x2 <2 (b) x1 +2 <2 (c) (i)? +)? <4 


3x, +X. =3 2) aie Oh, |x1| — x2 <0 
X1,X2 > 0 X1,X2 = 0 
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FIGURE 2.1 Feasible Set for the Crude Example 


Analysis: Applying the process of | 2.10 | produces the following plots: 


Xo ao) 
A ry 


3 3 feasible set 


feasible set 


(b) () 


Notice that the feasible set for system (b) is just the highlighted line segment, because 
one constraint is an equality. Also, system (c) allows negative variable values because 
the variable type is unrestricted. 


Graphing Objective Functions 


To find the best feasible point, we must introduce the objective function into a plot 
like Figure 2.1. Observe that the objective in our Two Crude example, call it 


C(xX1, X2) A 20x, + 15x (2.8) 


is a function of decision variables x, and x». Thus a plot requires a third dimension. 

Figure 2.2(a) depicts surface (2.8) in an x, versus x7 versus c coordinate system. 
For example, feasible solution x; = x. = 4 has value c(4, 4) = 20(4) + 15(4) = 140 
on the objective function surface. 


2.2 Graphic Solution and Optimization Outcomes 


cost 


200 - 


100 feasible set 


\ Nir Ase SN Sea N 
leeds AS ONT 16S > ern reeON 
(b) Two-dimensions with contours 


FIGURE 2.2 Graphic Solution of the Two Crude Model 


Because most of us have trouble visualizing, much less drawing 3-dimensional 
plots, mathematical programmers customarily employ the 2-dimensional alternative 
of Figure 2.2(b). There, the third dimension, cost, is shown through contours. 
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To introduce contours, we begin with any convenient point visible in the plot of 
the feasible space. For example, we might pick x; = 9,x2 = 0in the Two Crude model. 
Evaluating the objective function (2.8) at the point selected gives 


20(9) + 15(0) = 180 


thousand dollars. Thus all points on the contour where the objective function equals 
$180 thousand satisfy the equation 


20x; + 15x. = 180 


Plotting produces the straight-line contour marked “180” in Figure 2.2(b). 

With one typical objective function value such as 180 known, we complete the 
contouring by trying several similar objective function values. Figure 2.2(b) uses 
increments of $20 thousand obtained by plotting equations 


20x, + 15x2 = 100 
20x, + 15x2 = 120 


20x; + 15x = 240 


A contour exists for every possible objective value, so many other levels could 
have been chosen. We only wish to convey the trend and shape of the objective 
function. A small arrow perpendicular to the contours shows the direction in which 
the objective function improves. 


SAMPLE EXERCISE 23: PLOTTING OBJECTIVE FUNCTION CONTOURS | 


Show contours of each of the following objective functions over the fesdstisIe 1 region 
defined by y; + y2 < 2, y1, y2 > 0. 
(a) min 3y; + y2 (b) max 3y1 + y2 (c) max 2(y1)? + 2(2) 


Analysis: Contours for the three cases are as follows: 


(a) After identifying the feasible region by plotting the main and variable-type con- 
straints, we arbitrarily pick y; = 0, y2 = 1 to begin introducing the objective function 


2.2 Graphic Solution and Optimization Outcomes 


| as in principle |2.11]. The corresponding contour is the line where 
3y1 + y2 = 30) +) = 1 


Then other contours are introduced by incrementing the level to 2,3,...,5. 


(b) Contours for this maximize model are identical to those minimizing the same 
objective function in part (a). A small arrow on one of the contours reminds us of 
the direction in which the objective function improves. 


(c) For this more complex objective function, contours will not plot as straight lines. 
We begin with y; = 0, y2 = 1 and graph curve 


2(1)* + 2(y2)? = 2(0)? + 2(1)? = 2 


Then other contours arise from trying nearby levels 0.5, 4.5, and 8.0. 


Optimal Solutions 


Our goal in graphic analysis of an optimization modelis to identify an optimal solution 
if there is one. 


constraints. 


We have already determined that the feasible points in Figure 2.2(b) are the 
x, X2) in the shaded area. To identify an optimal solution, we examine the objective 
function contours. 


2.13_| Optimal solutions show graphically as poi 
function contour that intersects the FasOle gior 

Our Two Crude model is of minimize form, so the best objective contour is the 
lowest. We can identify an optimal solution by finding a feasible point on the lowest 
possible contour. 

Only a few contour lines are explicitly displayed in Figure 2.2(b), but infinitely 
many exist. It is clear from the pattern of those shown that the unique point satisfying 
condition is (x{,X5) = (2,3.5). The asterisk (*) or “star” denotes an optimal 
solution. 

This completes graphic solution of the Two Crude model. The optimal operat- 
ing plan uses 2 thousand barrels per day of Saudi petroleum and 3.5 thousand barrels 
per day of Venezuelan. Total daily cost will be 20(2)+15(3.5) = 92.5 thousand dollars. 


SAMPLE EXERCISE 2.4: SOLVING OPTIMIZATION MODELS GRAPHICALLY 


Return to the equipment yard model of Sample Exercise 2.1 and solve it graphically. 


Analysis: Identifying the feasible set and introducing objective function contours 
produces the following plot: 
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305 optimal €* = w* = 20 


20 E 


For this maximize problem we wish to be on the highest possible contour. Thus 
application of principle | 2.13 | yields optimal solution ¢* = w* = 20. 


Optimal Values 


Optimal solutions provide a best choice for decision variables and the optimal value 
is the corresponding objective function level. 


The ptimal value in an optimization modeli is the objective: -functi tion value _ 
any optimal solutions. a —r—~—“a—isO——C—C—~s~—sSsSsSS 


For example, the Two Crude optimum of Figure 2.2(b) has optimal value 
$92,500. 
Notice that two different objective values could not both be best. 


ave only one re optimal value. 


We see this graphically because there can only be one best contour level in 


Unique versus Alternative Optimal Solutions 


The optimal solution x} = 2, x} = 3.5 in our Two Crude example is also unique 
because it is the only feasible solution in Figure 2.2(b) achieving the optimal value. 
This does not always happen. Many models have alternative optimal solutions. 


2.16 | An optin 


alternative 0 


Still, all must have the same optimal value. 


We can illustrate principle very easily with another graphic solution. 


Suppose that crude prices change in the Two Crude example to produce objective 


2.2 Graphic Solution and Optimization Outcomes 


function 


min 20x, + 10x. 


Figure 2.3 shows the impact. There is still only one optimal value, $75 thousand. 
But there are now an infinite number of alternative optimal solutions along the high- 
lighted boundary of the feasible set; all lie on the optimal $75 thousand 
contour. 


eee Pieters | at more than 
solutions. 


* SOLUTIONS -. . S 


Determine otaphically which of the following optimization models has a a . unique 
optimal solution and which has alternative optima. 


(a) max 3w,+3w2 (b) max 3w,+w2 
st. wy + wo <2. st. wy tw 20) 
W1,W2 = 0 W1,W2 = 0 


Analysis: Graphic solution of these models is as follows: 


2 2 
alternative optimal : 
i solutions unique 
optimal 
1 bs 1 solution 
NEES ‘4 


Model (a) has alternative optimal solutions along the highlighted boundary, 
including (w1,w2) = (0,2), (wi,w2) = (1,1) and (w1, w2) = (2,0). Model (b) 
has unique optimal solution (w1,w2) = (2,0) because the optimal-value contour 
intersects the feasible space at only one point. 


Infeasible Models 


infeasible models have no optimal solutions. 
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FIGURE 2.3 Variant of the Two Crude Model With 
Alternative Optima 


Such models have no optimal solutions because they have no solutions at all. 

Infeasibility is also easy to illustrate with models small enough to be analyzed 
graphically. For example, consider modifying our Two Crude case so that only 
2 thousand barrels per day are available from each source. The resulting model is 


min 20x; + 15x (total cost) 

st. 0.3x, +0.4x2 > 2.0 (gasoline requirement) 
0.4x; + 0.2x. > 1.5 (jet fuel requirement) 
0.2x, + 0.3x2 > 0.5 (lubricant requirement) 
x1 2 (Saudi availability) 
X2 <2 (Venezuelan availability) 
X1,X%2. > 0 (nonnegativity) 


An attempt to graph the feasible space produces Figure 2.4. As before, each 
constraint corresponds to a line where it holds as an equality and a side of the line 
satisfying it as an inequality. This time, however, there are no (x1, x2) satisfying all 
constraints simultaneously. 


y no o point falling withi the 2 


V ODELS GRAPHICALLY 


Detoanine Pashley uhich of the following optimization models is feasible and 
which is infeasible. 
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FIGURE 2.4 Infeasible of the Two Crude Model 


| (a) max 3w,+ 2 (b) max 3w;+wo 

| s.t. W1 + W2 <2 s.t. wi + wo = 2 
| Witw. 21 Wwitw, 23 
| W1, W2 > 0 W1,W2 >0 


Analysis: Graphic solution of these models is as follows: 


| >W, 


(b) 


Model (a) has points in the shaded area that satisfy all constraints. Thus it is 
feasible. No points satisfy all constraints in model (b). That model is infeasible. 


Unbounded Models 


Another case where an optimization model has no feasible solution arises when it is 
unbounded. 
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‘model is unbounded when feasible choices of the dec 


ood objective function values. 


Unbounded models have no optimal solutions because any possibility can be im- 
proved. 

We can illustrate this outcome graphically with still another variant of the Two 
Crude case (although not a very realistic one). Suppose that Saudi Arabia decides to 
subsidize oil prices heavily so that Two Crude is paid $2 for each barrel it consumes, 
and further that Saudi Arabia will supply unlimited amounts of petroleum at this 
negative price. The result is a revised model 


min —2x, + 15x2 (total cost) 

st. 0.3x, +0.4x2 > 2.0 (gasoline requirement) 
0.4x, + 0.2x. > 1.5 (jet fuel requirement) 
0.2x1 +0.3x2 > 0.5 (lubricant requirement) 
x2 <6 (Venezuelan availability) 
X1,X2 > 0 (nonnegativity) 


Figure 2.5 shows a graphic solution. Notice that as x; (Saudi purchases) in- 
creases, we encounter feasible solutions with ever better objective function values. 
No solution is optimal because a better one can always be found. 

This is exactly what it means to be unbounded. 


FIGURE 2.5. Unbounded Variant of the Two Crude 
Example 


2.3 Large-Scale Optimization Models and Indexing 


DENTIFYING UNBOUNDED MODELS GRAPHICAL 


SAMPLE EXERCISE 2. 


Determine graphically which of the following optimization models has an Alea 
solution and which is unbounded. 


(a) max —3w,+ Ww» (b) max 3w;,+wo 
s.t. —W1 + WwW <1 s.t. —wytw <1 
W1,W2 > 0 W1,W2 = 0 


Analysis: Graphic solution of these models is as follows: 


(a) (b) 


Model (a) has a unique optimal solution at (w;, w2) = (0, 1). Model (b) is un- 
bounded because we can find feasible solutions on arbitrarily good contours of the 
objective function. 


2.3 LARGE-SCALE OPTIMIZATION MODELS AND INDEXING 


Because it could be solved graphically, our tiny Two Crude Petroleum example 
facilitated introduction of a variety of mathematical programming concepts. Still, it 
seriously misrepresents problem size. In real applications optimization models quick- 
ly grow to thousands, even millions, of variables and constraints. We begin to see 
how in this section and introduce the indexed notational schemes that keep large 
models manageable. 


EXAMPLE 2.2: Pi Hybrids 


To illustrate, we introduce our first real application of operations research.' A large 
manufacturer of corn seed, which we will call Pi Hybrids, operates ¢ = 20 facilities 
producing seeds of m = 25 hybrid corn varieties and distributes them to customers 
inn = 30 sales regions. They want to know how to carry out these production and 
distribution operations at minimum cost. 


‘Based on Mingjian Zuo, Way Kuo, and Keith L. McRoberts (1991), “Application of Mathematical 
Programming to a Large-Scale Agricultural Production and Distribution System,” Journal of the Opera- 
sonal Research Society, 42, 639-648. 
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After some effort, a variety of parameters that we will take as constant have been 
estimated: 


The cost per bag of producing each hybrid at each facility 

The corn processing capacity of each facility in bushels 

The number of bushels of corn that must be processed to make a bag of each hybrid 
e The number of bags of each hybrid demanded in each customer region 

e The cost per bag of shipping each hybrid from each facility to each customer region 


Our task is to produce a suitable optimization model. 


Indexing 


Indexes or subscripts permit representing collections of similar quantities with a 
single symbol. For example, 


(7. a. LOO} 


represents 100 similar values with the same z name, distinguishing them with the 
index i. 

Indexing is a powerful aid in keeping large-scale optimization models easy 
to think about and concise to write down. In fact, it often provides the initial model 
organization. 


22] The first step i in formulating a large optimization 1 model is to cho 
propriate indexes for the different dimensions of the problem. 


Our Pi Hybrids example has three main dimensions: facilities, hybrids, and 
sales regions. Thus we begin formulation of a model by assigning an index to each: 


f A production facility number (f = 1,...,£) 
h A hybrid variety number (A = 1,...,7™) 


r 4 sales region number (r = 1,...,7) 


Indexed Decision Variables 


We are now ready to think about what decisions Pi Hybrids’ problem requires. 
Clearly, they fall into two categories: how much to produce and how much to ship. 

There is a production decision for each plant and each hybrid. We could se- 
quentially number the corresponding decision variables 1, ..., km, but it would then 
be difficult to keep plant and hybrid issues separate. It is much more convenient to 
employ decision variables with distinct subscripts for plant and hybrids: 


Xfh = 4 number of bags of hybrid h produced at facility 
fileA ee 0m) 


Multiple subscripts are extremely common in large OR models. 


2. ao 
sion over v 


2.3 Large-Scale Optimization Models and Indexing 


Shipping at Pi Hybrid is distinguished by the production facility shipped from, 
the hybrid variety shipped, and the sales region shipped to. Following principle 


2.23, we complete the list of decision variables with a 3-subscript family: 


Y¢h,r & number of bags of hybrid h shipped from facility f to sales region r 
CSS TE yey eee, a) 
Notice that indexing has made it possible to define quite a large number of 
variables with just two main symbols. There are £m = 20(25) = 500 x-variables and 


émn = 20(25)(30) = 15,000 y-variables. The 15,500 total dwarfs the tiny examples 
of preceding sections, but it represents only a rather average size for applied models. 


Suppose that an vers eet i jis decision geht Wij.k,e, Where i and 
x range over 1,..., 100, while j and ¢ index through 1,...,50. Compute the total 
number of Meccion: variables. 


Analysis: The number of variables is 
(number of i) (number of j) (number of &) (number of £) 
= 100(50)(100) (50) 
= 25,000,000 


Indexed Symbolic Parameters 


if Pi Hybrids made only 2 products at 2 facilities, we could easily express total pro- 
duction cost as 


cost of cost of cost of cost of 

hybrid hybrid hybrid hybrid 

lat 11+] 2at xXi2+] Lat X21 +] 2at X2,2 (C2) 
facility facility facility facility 

1 1 2 2 


using the actual cost values. But writing out the ém = 20(25) = 500 production cost 
terms actually required in our model would be bulky and almost impossible to read 
or explain. 

The answer to this dilemma is more indexing, this time on input parameters. 


For example, after defining 
Pn & cost per bag of producing hybrid / at facility f 


»e may employ summation notation to express form (2.9) for any number of facilities 
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and hybrids as 


[So Wedlits 
Dah es Pf£hXfh 


(ie 


Moving indexes in the double sum capture terms for all combinations of facilities 
f=1,2,...,€@and hybridsh =1,2...,m. 


(a) Write the following sum more compactly with summation notation: 


2Wi,s + 2W2,5 + 2w3,5 + 2w4.s + 2ws5 


(b) Write out terms separately of the sum 
4 
yin 
i=1 


Analysis: 


(a) Introducing moving index i for the first subscript, the sum is 
5 5 
DL wis =2) | wis 
i=1 i=1 


(b) The 4 terms of the sum are 
1w; + 2w2 + 3w3 + 4w4 


Objective Function 
For similar reasons of convenience, define the following symbolic names for other Pi 
Hybrids input parameters: 


A corn processing capacity of facility f in bushels 


ug = 

a, 4 number of bushels of corn that must be processed to obtain a bag of 
hybrid h 

dn, & number of bags of hybrid A demanded in sales region r 


Sth,r & cost per bag of shipping hybrid h from facility f to sales region r 


With the aid of our indexed decision variables and these indexed parameters, 
we are now ready to formulate the objective function of a Pi Hybrids model. It 
should minimize 


total cost = total production cost + total shipping cost 


or 


[eis eo ip wie 


min >> >> ppaxgn+ >> >, >) SphrV ene 


fal h=1 f=! h=1 r=1 


2.3. Large-Scale Optimization Models and Indexing 


Indexed Families of Constraints 


Turn now to constraints for the Pi Hybrids model. What restrictions must decision 
variables satisfy? 

One family of constraints must enforce production capacities at the various 
facilities, Capacities uy are measured in bushels processed, with a, needed for each 
bag of hybrid h. Thus we may express the capacity requirement at each facility 1 by 


m_ / bushels bags of h 
2) ( per bag of ( produced < capacity at 1 
h=1 \h at 1 

or 


m 
ye ApX1n SU 
h=l 


There are € = 20 such capacity constraints for-different facilities. We could 
write each one down explicitly, but again, the model would become very bulky and 
hard to comprehend. 

Standard mathematical programming notation deals with this difficulty by list- 
ing indexed families of constraints. 


Written in this style, all capacity constraints of the Pi Hybrids model can be 
expressed in the single line 


m 
Yo anxpn < ur H(t Ale tee od 


Separate constraints are implied for f = 1,2,..., €. There are 20 in all. Equivalent 
forms are 


m 
SS anXth < ur for all f 
and 
Pe AnXth < uf Vf 
h=1 


>ecause we know that f ranges from 1 through @ and the mathematical symbol V 
means “for all”. 


An Peimeition model must decide how to allocate available suiipilies s; at sources 
= l,...,p to meet requirements 7; at customers j = 1,...,g. Using decision 
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variables 

wi; & amount allocated from source i to customer j 
formulate each of the following requirements in a single line. 
(a) The amount allocated from source 32 cannot exceed the supply available at 32. 
(b) The amount allocated from each source i cannot exceed the supply available at i. 
(c) The amount allocated to customer n should equal the requirement at 7. 


(d) The amount allocated to each customer j should equal the requirement at j. 


Analysis: 


(a) Only one constraint is required: 


n 
y W32,; < 532 
j=l 


(b) Here we require constraints for all sources i. Using notation , all m can be 
expressed: 


n 
Ses (imme lee eicg 
j=l 
(c) As with part (a) there is only one constraint: 
m 
se Win =!n 
i=l 


(d) This requirement implies constraints for each demand j. Using notation F 
all n can be expressed 


22 
(a) a 21,3 = b3 
=a 


22 
GY oz ts Oat 
i=1 
10 
(c) >. Zine = gs a lee A eye lee OU 
k=1 


Analysis: 


(a) This expression represents only 1 constraint, associated with fixed index 3. 


2.3. Large-Scale Optimization Models and Indexing 


(b) This expression represents 45 constraints, one for each p. 


(c) This expression represents 14(30) = 420 constraints, one for each i and each j. 


Pi Hybrids Example Model 


In terms of the notation we have defined, a full model for Pi Hybrids’ production-— 
distribution problem is 


Lie ih an 
min > ae PfhXth + My ye ye Sth,rYfhyr (total cost) 
fal h=l f=l h=l r=1 
m 
s.t. x AnXfh < ur ia llee ead (capacity) 
h=1 
£ 
ds Veni ane h=1,...,m; r=1,...,n (demands) (2.10) 
jo 


Yo yenr =e fae GaSe 7 (balance) 
jell 


Xn = 0 f=1,.-.4 h=1,...,m (aonmnesativity) 


Veh = O Pa ijsoegie) (= Wisconsin oaantt 


Besides the objective function and capacity constraints derived earlier, model 
(2.10) includes four new systems of constraints. The first of these enforces demand 
for each hybrid in each sales region. Amounts shipped from various facilities are 
summed to compute the total applicable to each demand. A second system of con- 
straints balances production and distribution. It requires that the total of any hybrid 
shipped from any facility should match the amount of that hybrid produced there. 
Finally, there are the variable-type constraints. The last two systems require all pro- 
duction and distribution quantities to be nonnegative. 


How Models Become Large 


The 500 x-variables and 15,000 y-variables of model (2.10) are subject to a total of 
20 + 750 + 500 + 15,500) = 16,770 constraints: 


E20 capacity constraints 
mn = 25(30) = 750 demand constraints 
£m = 20(25) = 500 balance constraints 


ém + €mn = 20(25) + 20(25)(30) = 15,500 nonnegativity constraints 


Stull. we have managed to write it compactly in only a few lines. 

In part this compactness derives from the power of indexed notation. But there 
s another reason. Model (2.10) actually involves only a few simple notions: produc- 
Son cost, shipping cost, capacity, demand, balance, and nonnegativity. What makes 
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it big is repetition of these notions over many combinations of facilities, hybrids, and 
sales regions. 
This is typical of the way that OR models grow. 


ainly by a relatively small number 
its being repeated many) times for 
and s so on. : 


2.26 | Optimization models 


With suitable indexing over such problem dimensions, we can express very large 
models in just a few lines. 


2.4 LINEAR AND NONLINEAR PROGRAMS 


Different classes of optimization models have enormously different tractability. This 
makes recognizing the major categories an important modeling skill. In this section 
we begin developing that ability by illustrating the fundamental distinction between 
linear programs and nonlinear programs. 


General Mathematical Programming Format 


The distinction begins with thinking of mathematical programs in terms of functions 
of the decision variables. 


22) | The general form of a mathematical program ¢ or x emule objective) opti- 
mization model is - 
min or max fa, i) 


St. gi(t1,..-, Xn) { 


WV EIA 
2 
™~ 
i 
aE 
= 


where f g1,....8m are given functions of decision variables x,...,%,, and — 
b,,..., Dm are specified constant parameters. 


Individual constraints may be of <, =, or > form. 
To illustrate, return to the Two Crude petroleum model of Section 2.1: 


min 20x; + 15x 

st. 0.3x; +0.4x%. > 2.0 
0.4x; + 0.2x2 > 1.5 
0.2x; + 0.3x2 > 0.5 


x1 <9 
XI <6 
X41, to 0 


Decision variables are x; and x2, so that nm = 2 and there are m = 7 constraints. In 
format | 2.27 | the implied functions are 


2.4 Linear and Nonlinear Programs 


f(%1,x2) A 20x, + 15x2 

gi (x1, X2) A 0.3x, + 0.4%. 
g2(X1, X2) A 0.4%, + 0.2x2 
23(%1, X2) 2 0.2x, + 0.3x9 


(2.11) 

ga(x1,X%2) 2 xy 

85(X1,%2) 2 x2 

86(%1, X2) S xy 

87(X1,%2) A x2 
Notice that there are g-functions for both main and variable-type constraints. 
Right-Hand Sides 
Format | 2.27 | collects everything involving the decision variables in the functions 
£21,.-++,8m- Constraint limits b;, ..., b,, must be constants. 


For obvious reasons these constraint constants b; are called the right-hand sides 
or RHSs) of the model. In our Two Crude example the right-hand sides are 


by = 2.0, bo = iss. b3 = 0.5, by => 9, bs => 6, be = 0, and by —Hi) 


mization model in general functional format and identify all required functions 
and right-hand sides: 


max (w1)? + 82 + (w3)? 
st. wy + 6w2 < 10+ w2 
(wo)? =7 
Wi = W3 
W1,W2 >0 
Wodeling: In format the objective function is 


fow1, w2, w3) & (1)? + Bw + (3)? 

After collecting all terms involving the decision variables on the left-hand side, con- 
straints have the form 

81(W1, W2, W3) < 10 

§2(W1, W2,W3) = 7 

83(W1, W2, w3) = 0 

84(W1, W2,W3) > 0 

85(W1, W2, W3) = O 
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where 
81(W1, W2, W3) 2 wi + 6w2 — w2 = wi + 52 
82(W1, W2, W3) & (w2)? 
93(W1, W2, W3) 2 w1 — w3 
g4(W1, W2,W3) Sw} 
85(W1, W2, W3) 2 w2 


Associated right-hand-side constants are 
by = 10, bo = Te b3 = 0, ba = 0, and bs =0 


Linear Functions 


We distinguish classes of mathematical programs according to whether functions 
fi 81, --+s 8m Of format are linear or nonlinear in the decision variables. 


2.28 | A function is linear if it is a constant-we 


Linear functions may involve only constants and terms with variables in the 
first power. For example, the objective function 


f(x, x2) & 20x) + 15x2 


of the Two Crude model is linear because it simply applies weights 20 and 15 in 
summing decision variables x; and x2. On the other hand, the objective function in 
Sample Exercise 2.12, 


f0w1, Wo, ws) A (1)? + 8w2 + (3)? 


is nonlinear. It includes second powers of some decision variables. 


SAMPLE EXERCISE 2.13: RECOGNIZING LINEAR FUNCTIONS | 


Assuming that x’s are decision variables and all other symbols are constant, ee 
mine whether each of the following functions is linear or nonlinear. 


(a) fx1, X23) & 9x1 — 17x3 
(b) fe, x2,%3) 2 Ye Gx; 
5) 
(C) (@t, %2,%3) ae = epic 3x2 — 6x3 


(d) ft, x2, x3) A xix. + (x2)? + Inxs) 
(e) fxr, x2, x3) A e%x, + In(B)x3 


(f) f@1,%2,%3) = ee 
X2 — X3 


Analysis: 


(a) This function is linear because it merely sums the 3 decision variables with weights 
9, 0, and —17, respectively. 


(b) This function is also linear because the c; are constants. 


2.4 Linear and Nonlinear Programs 


(c) This function is nonlinear because it involves negative powers of decision variable 
X41. 


(d) This function is nonlinear because it involves products, powers not 1, and loga- 
rithms of decision variables. 


(e) This function is linear. With a and £ constant, it is just a weighted sum of the 
decision variables. 


(f) This function is nonlinear because it involves a quotient, even though both nu- 
merator and denominator are linear functions. 


Linear and Nonlinear Programs Defined 


The functional forms in format 
programs. 


characterize linear programs and nonlinear 


Resiaiho that y’s are decision YARADIES and all other Gihels are constant, efeten 
mine whether each of the following mathematical programs is a linear program or a 
nonlinear program. 


a) min a(3y; + 1lya) 
5 
st. Di diy) = B 
j=1 


b) min a(3y; + 11y4)? 
5 


s.t. 2 diss SP 
et eee 
9 
ey imax. >; 


SC 
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Analysis: We apply definitions 


(a) This model is a linear program because the objective function and all constraints 
involve just weighted sums of the decision variables. 


(b) This model has the same linear constraints as the model of part (a). However, it 
is a nonlinear program because its objective function is nonlinear. 


(c) This model is a nonlinear program. Its objective function and last 9 constraints 
are linear, but the single nonlinear constraint 


yiy2 <= 100 


renders the entire model nonlinear. 


Two Crude and Pi Hybrids Models Are LPs 


Both the Two Crude Petroleum model of Section 2.1 and the Pi Hybrids model of 
Section 2.3 are linear programs. In the Two Crude case we exhibited the required 
functions in (2.11). Each obviously satisfies definition [2.28]. It is easier to be 
confused about the much larger Pi Hybrids model (2.10) because we assigned so many 
symbolic names to constants. However, a careful look will show that its objective 
function and every one of its constraints involve just a weighted sum of decision 
variables xf; and yep,,. It, too, is a linear program. 


EXAMPLE 2.3: E-MART 


For an example of a nonlinear program or NLP, consider the problem of budgeting 
advertising expenditures faced by a large European variety store chain we will call E- 
mart.? E-mart sells products in m = 12 major merchandise groups, such as children’s 
wear, candy, music, toys, and electric. Advertising is organized into n = 15 campaign 
formats promoting specific merchandise groups through a particular medium (cata- 
log, press, or television). For example, one variety of campaign advertises children’s 
wear in catalogs, another promotes the same product line in newspapers and mag- 
azines, while a third sells toys with television. The profit margin (fraction) for each 
merchandise group is known, and E-mart wishes to maximize the profit gained from 
allocating its limited advertising budget across the campaign alternatives. 


Indexing, Parameters, and Decision Variables for E-mart 
We begin a model by introducing indexes for the two main dimensions of the problem: 
g 4 merchandise group number (g = 1,...,m) 
c 4 campaign type number (c = 1,..., 7) 
Then we may denote major input parameters by 
Pg & profit, as a fraction of sales, realized from merchandise group g 
b A available advertising budget 


2Based on P. Doyle and J. Saunders (1990), “Multiproduct Advertising Budgeting,” Marketing 
Science, 9, 97-113. 


2.4 Linear and Nonlinear Programs 


Decisions to be made concern how to spend E-mart’s advertising budget. Thus 
we will employ decision variables 


x- 4 amount spent on campaign type c 


Nonlinear Response 


To complete a model, we must quantify how sales in each group g are affected by 
advertising expenditures on each campaign c. If the relationship is linear, 
sales increase 
in group g due | = Sy -X- 
to campaign c 
where 
Sec & parameter relating advertising expenditures in campaign c to sales growth in 
merchandise group g 


A linear form is attractive because it leads to easier analysis. 


Snes ne ones in optimization models = each at ae 
an optimization iE usaally reduces its Emetpility as oe to | 
forms. :  . 


Unfortunately, marketing researchers often find the linear alternative inappro- 
priate. The main difficulty is that linear functions produce what economists call equal 
returns to scale. 


2.32_| Linear functions implicitly assume that each unit increase in a decision 
variable has the same effect as the preceding increase: equal returns to scale. _ 


The E-mart experience shows something different. Their advertising history 
exhibits decreasing returns to scale; that is, each dollar of advertising in a particular 
campaign yields less than did the preceding dollar. 

Such unequal returns to scale imply nonlinearity. E-mart analysts chose the 
sonlinear form 

sales increase 
in group g due | = 5g log(x, + 1) (2.12) 
to campaign c 


This sales response function has the required decreasing returns because logarithms 
=row at a declining rate as x, becomes large. Adding +1 keeps the function nonneg- 
ative over x, > 0. 


E-mart Example Model 


After estimating constants sz, from history, we are ready to state the complete 
=-mart model: 
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Sec log(xe + 1) (total profit) 


n 


m 
max 2 Peg 


rail (=i 
n 2.13 
st. Xe <b (budget limit) es 
il 
X- > 0 c=1,...,n (nonnegative expenditures) 


The objective function maximizes total profit by summing sales increase expres- 
sions (2.12) times corresponding profit factors. A single main constraint enforces the 
budget limit, and variable-type constraints keep all expenditures nonnegative. The 
model is a nonlinear program because its objective function is nonlinear. 


2.5 DISCRETE OR INTEGER PROGRAMS 


Variables in mathematical programs always encode decisions, but there are many 
types of decisions. In this section we introduce discrete optimization models, which 
include decisions of a logical character qualitatively different from those of linear and 
nonlinear) programs. Discrete optimization models are also called integer (linear or 
nonlinear) programs, mixed-integer (linear or nonlinear) programs, and combinato- 
rial optimization problems. 


EXAMPLE 2.4; BETHLEHEM INGOT MOLD 


For an example, we turn to the problem confronted by Bethlehem Steel Corpora- 
tion in choosing ingot sizes and molds.’. In their process for making steel products, 
molten output from main furnaces is poured into large molds to produce rectangular 
blocks called ingots. After the molds have been removed, the ingots are reheated 
and rolled into product shapes such as I-beams and flat sheets. 

Bethlehem’s mills using this process make approximately n = 130 different 
products. The dimensions of ingots directly affect efficiency. For example, ingots of 
one dimension may be easiest to roll into I-beams, but another produces sheet steel 
with less waste. Some ingot sizes carinot be used at all in making certain products. 

A careful examination of the best mold dimensions for different products 
yielded m = 600 candidate designs. However, it is impractical to use more than 
a few because of the cost of handling and storage. We wish to select at most p = 6 
and to minimize the waste associated with using them to produce all n products. 


Indexes and Parameters of the Bethlehem Example 
Our Bethlehem problem has two major index dimensions: 
i A mold design number (i = 1, ..., ™) 
j & product number (j = 1,...,7) 
One set of input parameters are the 
ci; & amount of waste caused by using ingot mold i on product j 


The other input required for a model is some indication of which products can use 


3Based on EF J. Vasko, F. E. Wolf, K. S. Stott, and J. W. Scheirer (1989), “Selecting Optimal Ingot 
Sizes for Bethlehem Steel,” Interfaces, 19:1, 68-84. 


2.5 Discrete or Integer Programs 


which molds. For this purpose we define index sets 


jj 4 collection of indexes i corresponding to molds that 
could be used for product j 


Ifi ¢ J;, mold i is feasible for product j. 


Discrete versus Continuous Decision Variables 


As usual, actual modeling begins with decision variables. Notice, however, that the 
decisions to be taken in our Bethlehem example are qualitatively different from 
those of earlier models. A mold design will either be selected or not. Once selected, 
it will either be used for a given product, or it will not. 

Such decisions require a new, logical, discrete variable type. 


2.33 | A variable i is discrete if it is limited to a fixed « or Fcountable set of values. 
Often, the choices are only and L 


Frequently occurring discrete variable types include binary or 0-1 variables 
limited to values 0 and 1, and nonnegative integer variables that may take any non- 
negative integer value. 

We employ discrete variables mainly to model decisions of an all-or-nothing, 
either-or character such as those involved in our Bethlehem ingot mold example. In 
particular, we will use 


0 if not 


Such y; are discrete because they are limited to two values. We ee value 1 to mean 
an event occurs, and value 0 when it does not. A value of 0.3 or = + has no physical 
meaning. 

In a similar manner, we model decisions about which mold to use in making 
each product with decision variables 


: | 1 if ingot mold i is selected 
i= 


A 

PO me 

2 Ohl hitnot 

Again the variables are allowed only a countable set of values to reflect the logical 
character of the decisions. 

Contrast these decision variables with the continuous variables we encountered 


in earlier models. 


| 1 if ingot mold i is used for product j 


2.34 | A variable is continuous if it can take on any value in a specified interval. 


For example, a nonnegative variable is continuous because it can assume any value 
in the interval [0, -++oo). 

Variables in the Two Crude model of Section 2.1 certainly have this continu- 
ous character. They are denominated in thousands of barrels of petroleum, and any 
nonnegative real value has a physical interpretation. Technically speaking, variables 
of the Pi Hybrids model in Section 2.3 should be limited to the integers (a discrete 
set) because they count bags of corn seed produced and shipped. Still, the num- 
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bers involved are likely to be so large that there is no loss of validity in allowing all 
nonnegative values. We will see in later chapters that there is a considerable gain in 
tractability. 


continuous Yanai are 
~ Variables. — 


"VARIABLES 


Decide whether a discrete or a contin ons variable would be best employed to model 
each of the following quantities, 


(a) The operating temperature of a chemical process 

(b) The warehouse slot assigned a particular product 

(c) Whether a capital project is selected for investment 

(d) The amount of money converted from yen to dollars 

(e) The number of aircraft produced on a defense contract 

Modeling: 

(a) A temperature can assume any value ina physical range. Itis naturally continuous. 


(b) The slot will be selected from a finite, and thus countable, list. It will probably 
require a discrete variable. 


(c) Assuming that one cannot invest in just part of a project, there are only two 
possibilities: Take the project or reject it. A 0-1 discrete variable is appropriate. 


(d) Amounts of money can assume any nonnegative value. They should be modeled 
with continuous variables. 


(e) If the number of aircraft is likely to be large, principle | argues for a contin- 
uous variable even though the number of aircraft is clearly atiniable If only a few 
expensive aircraft are planned, it may be necessary to model discreteness explicitly. 
Then a nonnegative integer variable is required. 


Constraints with Discrete Variables 


We are now ready to begin forming the constraints of our Bethlehem ingot mold 
example. One requirement is that at most p molds be selected: 


m 
LM SP 


Notice that our convention of making the variable = 1 when something happens, 
and = 0 otherwise makes such requirements easy to express. 


2.5 Discrete or Integer Programs 


A similar requirement is that each product be assigned exactly one mold design. 
We may sum over the possible choice to express those constraints as 


Deel Tea eel 


iel; 
Notice again how easy it is to model “at least 1,” “at most 1,” and “exactly 1” restric- 
tions with 0-1 discrete variables. 
A last system of main constraints for the Bethlehem model must encode the 
dependency between x and y variables. Mold i cannot be assigned to product j unless 
it is one of the p selected. That is, 


Mes Yi; (iceman RRR Tay ferme le ene 7) 


SAMPLE EXERCISE 2.16: EXPRESSING CONSTRAINTS IN 0-1 VARIABLES 
In choosing among a collection of 16 investment projects, variables 


Wi = a 
u 0 otherwise 


A | il if project j is selected 

Express each of the following constraints in terms of these variables. 
(a) At least one of the first eight projects must be selected. 

(b) At most three of the last eight projects can be selected. 

(c) Either project 4 or project 9 must be selected, but not both. 


d) Project 11 can be selected only if project 2 is also. 


Modeling: 
8 16 
(a) So wel (b) )5 wj <3 
j=l j=9 
(c) W4+wWo =1 (d) wit < wo 


Bethlehem Ingot Mold Example Model 


We may complete a discrete optimization model of our Bethlehem ingot mold ex- 
ample as 


min Le a CrjXiy (total waste) 
j=l ie]; 
mM 
s.t. = yi < 2p (select at most p) 
i=1 
oe ee ule ence (one each product) (2.14) 
ie]; 
He eS f= Tite le (use only if selected) 
Ve —Osoralee ras tall erste iM (binary variables) 


a —=Vorls- fj = los, eed 
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The objective function of this model merely totals the scrap waste associated 
with assigning molds to products. In addition to the main constraints formulated 
above, we have included m variable-type constraints on the yj and mn such constraints 
on the x;;. The specification “= 0 or 1” signals that these variables are discrete and 
may take on only the values 0 and 1. 


Integer and Mixed-Integer Programs 


A mathematical program is a discrete optimization model if it includes any discrete 
variables at all. Otherwise, it is a continuous optimization model. 

Discrete models are often called integer programs because we may think of 
discrete variables as being restricted to integers within an interval. For example, 


yj =Oorl 
is equivalent to 

O<yj <1 

y; integer 


Whenever the allowed values of a variable are countable, they can be aligned with 
the integers in a similar way. 


2.36_| An optimization model is an integer program (IP) if any one of its deci- _ 
sion variables is discrete. If all variables are discrete, the model is a pure inte 
program; otherwise, it is a mixed-integer program. — 


(a) w; = 0, T= dpeeng 
(b) w; =Oor1, jan Sp 
Wp+1 = O and integer 


(c) w; = 0, Jy ea 
Wp+1 = 0 and integer 


Analysis: We apply definitions : 
(a) Here all variables are continuous, so the model is continuous, not discrete. 


(b) The first p variables are limited to 0 and 1, and the last to any nonnegative integer. 
Thus the model is a pure integer program. 


(c) The one integer variable makes this model discrete, and so an IP. It is a mixed- 
integer program because it also has continuous decision variables. 


2.5 Discrete or Integer Programs 


Integer Linear versus Integer Nonlinear Programs 


2.29 jofalinear program except 


Bethlehem ingot model (2.14) would fulfill definition 
for the binary type of its variables. Thus it is natural to classify it an integer linear 
program. 


(LP) a its aides objective function and all main constraints are e linear. 


The alternative is an integer nonlinear program. 


Te that all W; are decision variables, detains maieiiee ean GE ie fellow 
ing mathematical programs is best described as a linear program (LP), a nonlinear 
program (NLP), an integer linear program (ILP), or an integer nonlinear program 
(INLP). 


(a) max 3w; + 14w2 —-w3 
S.t. wW1<wW2 
W, + W2+w3 = 10 
w; =Oorl hile 


(b) min 3w; + 14w2 — w3 
s.t. W1W2 <1 
W1+W2+w3 = 10 
w; = 0 of a rece Hie: 
w, integer 
In(w2) 


c) min 3w,+9 
W3 
s.t. Wi <W2 


wi +w2+w3 = 10 
W2,W3 >1 
Ww; >0 


d) max 19w, 
s.t. Wi < Wo 
WwW, +W2+w3 = 10 
W2,W3 = af 
W1= 0 
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Analysis: 


(a) Except for its discrete variable-type constraints, this model would be a linear 
program because the objective function and both main constraints are linear. Thus 
the model is an integer linear program (ILP). 


(b) The product in its first main constraint makes this model nonlinear. However, it 
would not usually be called a nonlinear program because w is discrete. The model 
is best classified as an integer nonlinear program (INLP). 


(c) The logarithm and quotient terms in its objective function make this model nonlin- 
ear. Since all variables are continuous, it should be classified as a nonlinear program 
(NLP). 


(d) The objective function and all main constraints of this model are linear and 
variable types are all continuous. Thus the model is a linear program (LP). 


EXAMPLE 2.5: PURDUE FINAL EXAM SCHEDULING 


We may illustrate integer nonlinear programming applications with a problem fa- 
miliar to every college student: final exam scheduling. In a typical term Purdue 
University* picks one of n = 30 final exam time periods for each of over m = 2000 
class units on its main campus. Most exams involve just one class section, but there 
are a substantial number of “unit exams” held at a single time for multiple sections. 

The main issue in this exam scheduling is “conflicts,” instances where a student 
has more than one exam scheduled during the same time period. Conflicts burden 
both students and instructors because a makeup exam will be required in at least one 
of the conflicting courses. Purdue’s exam scheduling procedure begins by processing 
enrollment records to determine how many students are jointly enrolled in each pair 
of course units. Then an optimization scheme seeks to minimize total conflicts as it 
selects time periods for all class units. 


Indexing, Parameters, and Decision Variables 
for Purdue Finals Example 


As usual, we begin a model of this problem by introducing indexes for its main 
dimensions: 


i A class unit number (i = 1,..., ™) 
t A exam time period number (t = 1,...,7) 
Then discrete decision variables encode the schedule options: 
eile if class 7 is assigned to time period t 
Xit = : 
3 0 otherwise 
Also, we define joint enrollment input parameters: 


e; 2 number of students taking an exam in both class unit i and class unit i’ 
Based on C. J. Horan and W. D. Coates (1990), “Using More Than ESP to Schedule Final Ex- 


ams: Purdue’s Examination Scheduling Procedure II (ESP II),” College and University Computer Users 
Conference Proceedings, 35, 133-142. 


2.6 Multiobjective Optimization Models 


Nonlinear Objective Function 


The main challenge in modeling Purdue’s final exam scheduling problem with the 
foregoing notation is to represent total conflicts in an objective function. To begin, 
focus on any pair of courses 7 and i’. The product 


1 if i and 7’ are both scheduled at time period ¢ 
XitXit = : 
ages 0 if not 


Thus we may sum over time periods and multiply by joint enrollment to express the 
conflict for any pair: 
n 
conflicts between i andi’ = e;; DS XitXirt 
t=1 
It remains only to total such expressions over all course pairs i, i’. The result is 
the Purdue final exam scheduling objective function: 


m-1 m n 
min a ye ej7 > XitXit (total conflicts) (2.15) 
i=l f=it1 t= 
Notice that summations have been indexed so that i’ > i, to avoid counting any pair 
more than once. The first sum considers all i except the last, which has no higher 
index, and the second adds in all pairs with 7’ > i. 


Purdue Final Exam Scheduling Example Model 
Beginning from objective function (2.15), we may model Purdue’s problem: 


m-l1 m n 


min * > enn) XieXe.t (total conflicts) 


i=1 /=i+1 t=1 
n 


2.16 
s.t. :¥ all i=1,...,m (class i scheduled) ( ) 
t=1 


Xi2=Oorl fired Lee {pam ay) 


Main constraints simply assure that each class unit i is assigned exactly one exam 
time period ¢. 

Model (2.16) is an integer nonlinear program (INLP). Its main constraints are 
linear, but the objective function has nonlinear product terms and variable types 
discrete. 


2.6 MULTIOBJECTIVE OPTIMIZATION MODELS 


All the models considered so far have a clear, quantitative way to compare feasible 
solutions. That is, they have single objective functions. In many business and indus- 
trial applications, single objectives realistically model the true decision process. Such 
organizations often are really satisfied to maximize some measure of profit or mini- 
mize some approximation to cost, although other objectives may also be relevant. 
Matters become much more confused when the problem arises in the govern- 
ment sector, or in a complex engineering design, or in circumstances where uncer- 
tainty cannot be ignored. In such applications, solutions may be evaluated quite 


59 


60 Chapter Two e Deterministic Optimization Models in Operations Research 


differently by different participants in the decision process, or against different per- 
formance criteria. None can be discounted. A multiobjective optimization model 
is required to capture all the perspectives—one that maximizes or minimizes more 
than one objective function at the same time. 


EXAMPLE 2.6: DUPAGE LAND USE PLANNING 


Perhaps no public-sector problem involves more conflict between different interests 
and perspectives than land use planning. That is why a multiobjective approach was 
adopted when government officials in DuPage County, Illinois, which is a rapidly 
growing suburban area near Chicago, sought to construct a plan controlling use of 
its undeveloped land. 


Table 2.1 shows a simplified classification with m = 7 land use types. The 


problem was to decide how to allocate among these uses the undeveloped land in 
the county’s n = 147 planning regions. 


TABLE2.1_ Land Use Types 
in DuPage Example 


~ 


Land Use Type 


Single-family residential 
Multiple-family residential 
Commercial 

Offices 

Manufacturing 

Schools.and other institutions 
Open space 


NADAS WNH 


Multiple Objectives 


No single criterion fully captures the appropriateness of assigning undeveloped acres 
to a given use. DuPage analysts employed five: 


1. 


2. 


Compatibility: an index of the compatibility between each possible use in a region and 
the existing uses in and around the region. 


Transportation: the time incurred in making trips generated by the land use to/from 
major transit and auto links. 


Tax load: the ratio of added annual operating cost for government services associated 
with the use versus increase in the property tax assessment base. 


Environmental impact: the relative degradation of the environment resulting from the 
land use. 


Facilities: the capital costs of schools and other community facilities to support the land 
use. 


A good plan should make the first of these objectives large and the others small. 


Assigning the indexes 


i A land usé type (i = 1,..., m) 


j 4 planning region (j = 1,...,7) 


the following symbolic constants parameterize the five objectives: 


5Based on Deepak Bammi and Dalip Bammi (1979), “Development of a Comprehensive Land Use 


Plan by Means of a Multiple Objective Mathematical Programming Model,” Interfaces, 9:2, part 2, 50-63. 


2.6 Multiobjective Optimization Models 


ci; & compatibility index per acre of land use i in planning region j 


t,; & transportation trip time generated per acre of land use i in planning region j 


r,; & property tax load ratio per acre of land use i in planning region j 


A 
ij 


relative environmental degradation per acre of land use i in planning region j 


fi; & capital costs for community facilities per acre of land use i in planning region j 


Then with nonnegative decision variables 


xi; 4 number of undeveloped acres assigned to land use i in planning region j 


we have the following multiple objectives: 


max 


min 


min 


min 


Constraints of the DuPage Land Use Example 


Some of the main constraints enforced in the DuPage example are straightforward. 


Define symbolic parameters 
b; 


b 


number of undeveloped acres in planning region j 


county-wide minimum number of acres allocated to land use type / 


rocky areas, etc. 


& 
u; 4 county-wide maximum number of acres allocated to land use type i 
A 


number of acres in planning region j consisting of undevelopable floodplains, 


Then a first system of main constraints assures that all undeveloped land in each 


planning region is allocated: 


m 


ey ff aes ll 
i=! 


Two others enforce county-wide lower and upper limits on various uses: 


n 
) xij = i pe. me 
j=l 


n 
) Xij S Ui lhe eed 
j=l 
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Finally, we need to assure all undevelopable land is assigned to parks and other open 
space: 
X77 2 Oj jen 


The more complex constraints describe how land uses interact. In particular, 
single- and multiple-family residential development creates a demand for nearby 
commercial centers and open space, as well as new schools and other institutions. 
Using the symbolic parameters 


s; 2 new acres of land use i implied by allocation of an acre of undeveloped land 
to single-family residential 


d; & new acres of land use i implied by allocation of an acre of undeveloped land 
to multiple-family residential 


we obtain 

Xij = SiX1,j + djx2,; (iS Opals lt 
Acres in i = 3, commercial, i = 6, institutions, and i = 7, open space, must meet the 
implied demands. 


DuPage Land Use Example Model 


Collecting all the above produces the following multiobjective optimization model 
of our DuPage example: 


m n 
max iz CijXij (compatibility) 
tL, j= 
m n 
min SO) Hx (transportation) 
i=l j=l 
m n 
min D. 5" TijXij (tax load ratio) 
i=1 j=l 
m n 
min Sa C; jXij (environmental) 
i=1 j=l 
m1 n 
min San Sfij*ij (facilities) 017 
i=l j=l (2.17) 
m a 
s.t. Des Xig = dD; jie em (all used) 
i=l 
A 
Ss logge li jer Men 7 (use minimums) 
j=l 
n 
a Xi SU; ad eld (use maximums) 
j=l 
Bag 20; Hi ly bi (undevelopable) 


Xij > SjX1,j + dix2,j i= oy 6, de J] = (ie seers (implied needs) 
xij >0 i=1,....m; j=1,...,n (nonnegativity) 


The only new element is nonnegativity type-constraints on the decision variables. 


2.7 Classification Summary 


Conflict among Objectives 
Like so many other applications, DuPage’s land use planning could be modeled 
validly only with multiple objective functions. Still, we will see in later chapters 
that there is a price in tractability. It is almost certain that the objective functions 
will conflict about the best allocation of land uses. For example, a solution placing 
trip-generating manufacturing land uses near traffic arteries will score well on the 
transportation objective, but it may not be at all compatible with existing land use, 
and it may severely degrade sensitive environments. 

Single objective models are easier to deal with because they avoid such conflicts. 
With multiple objectives it is not even clear how to define an “optimal” solution. 


“fepred to munldobreeite ones bec conflicts among objectives usually make 
multiobjective models less tractable. | 


SAMPLE EXERCISE 2.19: UNDERSTANDING MULTIPLE-OBJECTIVE CONFLICT 


Consider the multiobjective mathematical program 
max 3z; +22 
min wezesaze 
st. Zp +2z2 <3 
21,2220 
Graph the feasible region and show that the best solutions for the two objective 
functions conflict. 


Analysis: The feasible region and contours of the two objective functions are as 
follows: 


max 3z1 + Z2 


Feasible solution z; = 3, z2 = 0 is optimal for the first objective function, but z; = 0, 
>> = 3 is optimal for the second. To find a solution that is overall “best,” the analysis 
must somehow balance these conflicting objectives. 


7 CLASSIFICATION SUMMARY 


ver and over in the rest of this book we will see the importance of knowing whether 
» model is linear or nonlinear, continuous or discrete, single or multiobjective, when 
termining how tractable it will be and what methods are appropriate for trying to 
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solve it. One of the most important skills for any practitioner of operations research 
is the ability to recognize these distinctions and know their implications. 

To clarify the classification, Figure 2.6 provides a summary in a single display. 
The most tractable linear programming (LP) case has continuous variables, linear 
constraints, and a single linear objective function. If either constraints or the objec- 
tive are nonlinear, it becomes a nonlinear program (NLP). Any discrete variables 
turn LPs into integer linear programs (ILPs) and NLPs into integer nonlinear pro- 
grams (INLPs). All forms become more complex if there is more than one objective. 


multiple 


single 

, nonlinear 

single objective 
linear 

objective 


linear 
constraints 


nonlinear 
constraints 


discrete 
variables variables 


objective 


continuous 


FIGURE 2.6 Classification of Optimization Models 


EXERCISES 


2-1 The Notip Table Company sclls two models of 
its patented five-leg tables. The basic version uses a 
wood top, requires 0.6 hour to assemble, and sells for 
a profit of $200. The deluxe model takes 1.5 hours 
to assemble because of its glass top, and sells for a 
profit of $350. Over the next week the company has 
300 legs, 50 wood tops, 35 glass tops, and 63 hours 
of assembly available. Notip wishes to determine 
a maximum profit production plan assuming that 
everything produced can be sold. 


(a) [Q] Formulate a mathematical programming 
model with 4 main constraints to select an op- 
timal production plan using decision variables 
x, 4 number of basic models produced and 
x2 & number of deluxe models. 

[6] & Enter and solve your model with the class 
optimization software. 

Using a two-dimensional plot, solve your model 
graphically for an optimal product mix, and ex- 
plain why it is unique. 

[6] On a separate two-dimensional plot, show 
that that the model has alternative optimal so- 
lutions if profits are $120 and $300, respectively. 


(b) 
(c) 


(d) 


2-2 Wiley Wiz is a mutual fund manager trying to 
decide how to divide up to $12 million between do- 
mestic and foreign stocks. Domestic stocks have 
been returning 11% per year and foreign 17%. Nat- 
urally, Wiley would like to maximize the annual 
return from his investments. Still, he wants to exer- 
cise some caution. No more that $10 million of the 
fund should go into domestic stocks and no more 
than $7 million into foreign. Also, at least half as 
much should be invested in foreign as domestic, 
and at least half as much in domestic as foreign to 
maintain some balance. 


(a) Formulate a mathematical programming model 
with 5 main constraints to decide Wiley’s op- 
timal investment plan using decision variables 
x; £ millions of dollars invested in domestic 
stocks and x2 4 millions of dollars invested in 
foreign stocks. 

6] [Enter and solve your model with the class 
optimization software. 


(b) 
(©) 


Using a two-dimensional plot, solve your model 
graphically for an optimal investment plan. 


(d) On a separate two-dimensional plot, show 
graphically that the problem has alternative op- 
timal solutions if rates of return are equal for 
domestic and foreign. 


2-3 The Tall Tree lumber company owns 95,000 acres 
of forestland in the Pacific northwest, at least 50,000 
of which must be aerially sprayed for insects this year. 
Up to 40,000 acres could be handled by planes based 
at Squawking Eagle, and up to 30,000 acres could 
be handled from a more distant airstrip at Crooked 
Creek. Flying time, pilots and materials together 
cost $3 per acre when spraying from Squawking Ea- 
gle and $5 per acre when handled from Crooked 
Creek. Tall Tree seeks a minimum cost spraying 
plan. 


(a) Formulate a mathematical programming model 
to select an optimal spraying plan using decision 
variables x; 4 thousands of acres sprayed from 
Squawking Eagle and x, 4 thousands of acres 
sprayed from Crooked Creek. 

(b) EQ] Enter and solve your model with the class 
optimization software. 

(c) Using a two-dimensional plot, solve your model 
graphically for an optimal spraying plan, and ex- 
plain why it is unique. 

(d) On a separate two-dimensional plot, show 
graphically that the problem is unbounded if the 
Squawking Eagle capacity and both nonnegativ- 
ity constraints are omitted. 

e) On a separate two-dimensional plot, show 
graphically that the problem becomes infeasi- 
ble if the Crooked Creek facility is destroyed 
by fire. 


2-4 The Fast Food Fantasy (Triple-F) hamburger 
chain is attempting to respond to customer de- 
mand for more healthy food by introducing a new 
birdburger made from a combination of beef and 
chicken. The new burger should weight at least 
125 grams and have at most 350 calories, 15 grams 
of fat, and 360 milligrams of sodium. Each gram 
of beef used has 2.5 calories, 0.2 gram of fat, and 
3.5 milligrams of sodium. Corresponding values for 
chicken are 1.8 calories, 0.1 gram, and 2.5 milligrams. 
Triple-F wants to find the mix that will meet all re- 
quirements and maximize beef content. 


a) Formulate a mathematical programming model 
to decide an optimal birdburger blend using de- 
cision variables x, 4 grams of beef per burger 
and x2 A grams of chicken per burger. 


Exercises 65 


(b) © Enter and solve your model with the class 
optimization software. 

(c) Using a two-dimensional plot, solve your model 
graphically for an optimal ingredient mix, and 
explain why it is unique. 

(d) On a separate two-dimensional plot, show 
graphically that the model becomes infeasible 
if weight requirement is raised from 125 grams 
to 200 grams. 

(e) On a separate two-dimensional plot, show 
graphically that the model is unbounded if the 
minimum weight and nonnegativity constraints 
are omitted. 


2-5 Sun Agriculture (SunAg) operates a farm of 
10,000 acres in the dry southwestern part of the 
United States. In the next season SunAg can plant 
acres in either vegetables, which return a profit of 
approximate $450 per acre, or cotton, which returns 
$200 per acre. As a precaution against bad weather, 
insects, and other factors, SunAg will plant no more 
than 70% of its total holdings in any one of these 
options. Also, irrigation water is limited. To grow 
vegetables requires 10 units (of water) per acre, and 
cotton requires 7, out of a government allocation of 
70,000 units per season. SunAg wishes to develop a 
planting plan that maximumes profit. 


(a) Formulate a mathematical programming model 
to solve SunAg’s problem using 2 main con- 
straints, 2 upper bound constraints, 2 variable- 
type constraints, and decision variables vy = 
acres in vegetables and c = acres in cotton. 

(b) [4] SEnter and solve your model with the class 
optimization software. 

(c) Using a two-dimensional plot, find an optimal 
plan graphically. 

(d) Show graphically that if we accidentally omit- 
ted the upper and lower bound constraints on 
the two variables, the resulting model would be 
unbounded. 

(e) Now return to the correct model and imagine 
that SunAg is required by government regula- 
tion to plant all its acres. Show graphically that 
this case is infeasible. 


2-6 The Kazak Film company needs to cut 15 long 
tolls and 10 short rolls of film from stock pieces. 
Each stock piece can be cut in one of two patterns. 
The first produces 5 long and 2 short rolls; the second 
yields 3 long and 5 short. Once any part of a piece 
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of stock is cut, anything that remains is scrap. Also, 
neither pattern should be used more than 4 times 
because the jig used to cut it will become too inaccu- 
rate. Kazak wants to find the allowable combination 
of patterns that will minimize the number of stock 
pieces required. 


(a) [| Formulate a mathematical model to decide 
what patterns to use. Use decision variables 
x, & number of times pattern 1 is used and x2 4 
number of times pattern 2 is used. 

(b) [§] Both variables in your model should be re- 
stricted to take on only integer (whole number) 
values. Explain why. 

(c) [] Enter and solve your model with class op- 
timization software. 

(d) Using a two-dimensional plot, solve your model 
graphically for an optimal cutting plan. Re- 
member to consider only integer points. 

(e) Explain how your plot shows that the model has 
alternative optimal solutions. 


2-7 A factory is building a 500-square feet open rect- 
angular cooling pool for water exhausted from its 
main process. The pool will be 8 feet deep, its length 
should be at least twice its width, and there is room 
for a width of at most 15 feet. We wish to choose the 
feasible design that minimizes cost by minimizing the 
concrete area of the pool walls. 


(a) [Q] Formulate a mathematical programming 
model with 3 main constraints to choose an op- 
timal design using decision variables x; 4 length 
of the pool and x2 4 width of the pool. 

(b) [6] Enter and solve your model with the class 
optimization software. 

(c) Using a two-dimensional plot, solve your model 
graphically for an optimal design. 

(d) [§] On a separate two-dimensional plot, show 
graphically that the problem becomes infeasi- 
ble if the pool can be at most 25 feet in length. 


2-8 An architect is designing a cylindrical hotel that 
will have 150,000 square feet of floor space. She 
wishes to make the hotel have as many 10-feet-high 
floors as possible, but the height of the building 
should not exceed 4 times its diameter or it might 
be unstable. (Fractional numbers of floors are ac- 
ceptable in this rough analysis.) 


(a) Formulate a mathematical programming model 
with 2 main constraints to choose an optimal 
design using decision variables x, 4 diameter of 
the hotel in feet and x. 4 number of floors. 


(b) | Enter and solve your model with the class 
optimization software. 

(c) Using a two-dimensional plot, solve your model 
graphically for an optimal design. 

(d) On a separate two-dimensional plot, show 
graphically that the problem becomes infeasi- 
ble if the diameter is limited to 50 feet. 


2-9 Suppose that an optimization model in decision 
variables w; and w> has constraints 


5w, + 2w. < 10 


W2 > 0 


(a) [§] Devise a maximizing linear objective func- 
tion for which the model has a unique optimal 
solution, and demonstrate that fact by solving 
the model graphically. 

(b) [Q] Devise a maximizing linear objective func- 
tion for which the model has multiple optimal 
solutions, and demonstrate that fact by solving 
the model graphically. 

(c) [S| Devise a maximizing linear objective func- 
tion for which the model is unbounded, and 
demonstrate that fact by solving the model 
graphically. 


2-10 Suppose that an optimization model in decision 
variables w; and w> has constraints 


—w1 +W2 <7 


w,>0 


(a) Devise.a minimizing linear objective function 
for which the model has a unique optimal solu- 
tion, and demonstrate that fact by solving the 
model graphically. 

(b) Devise a minimizing linear objective function 
for which the model has multiple optimal solu- 
tions, and demonstrate that fact by solving the 
model graphically. 

(c) Devise a minimizing linear objective function 
for which the model is unbounded, and demon- 
strate that fact by solving the model graphically. 


2-11 Write each of the following as compactly as 
possible using summation and “for all” indexed no- 
tation. 


(a) E&] min 3y31 + 3ys2 +4ya1 + 4ya2 
(b) max 1y3 + 2y3,2 + 3y3,3 + 43,4 

(c) BY] max ayyi4 + o2yr4 +--+ + OpYp4 
(d) min fiy1 + Boy2 +--+ + Bryn 


(e) A ywity2tnatya=o 
y2,1 1 Y2,2 + Y2,3 + 2,4 = S2 
y3,1 + 3,2 + y3,3 + 3,4 = 53 


(f) diay + a1 2y2 + a13¥3 + days = by 
42,191 + 42,22 + 2,393 + doays = bo 
43,191 + 43,22 + 43,393 + d3,4y4 = bg 


2-12 Suppose that the decision variables of a math- 
ematical programming model are 


xij, 4 amount of product i 
produced on manufacturing 
line j during week t¢ 


Wi hevt— all reer loa ele eer Os eo ere uiaaSe 
summation and “for all” indexed notation to write 
expressions for each of the following systems of con- 
straints in terms of these decision variables, and de- 
termine how many constraints belong to each sys- 
tem. 


(a) [] Total production on any line in any week 
should not exceed 200. 

(b) EK] The total 7-week production of product 5 
should not exceed 4000. 

(c) &] Atleast 100 units of each product should be 
produced each week. 


2-13 & Repeat Exercise 2-12, this time coding the 
variables and constraints in the language of the class 
optimization software. 


2-14 Suppose that the decision variables of a math- 
ematical programming model are 


Xi & acres of land plot i allocated 
to crop] in year t 


muene my il ela fj eee. 0) tf et 10! 
Use summation and “for all” indexed notation to 
write expressions for each of the following systems 
of constraints in terms of these decision variables, 
and determine how many constraints belong to each 
system. 


a) The acres allocated in each plot i cannot exceed 
the available acreage (call it a;) in any year. 

(b) Atleast 1000 total acres must be devoted to corn 
(crop j = 4) in each year. 

c) At least one-third of the total acreage planted 
over the 10 years must be in soybeans (crop 
i= 2). 


2-15 ©! Repeat Exercise 2-14, this time coding the 
variables and constraints in the language of the class 
optimization software. 
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2-16 Assuming that decision variables are y,,..., y3, 


identify the objective function f, constraint functions 
gi, and right-hand sides b; of the general mathe- 


matical programming format corresponding 
to each of the following optimization models. 


(a) EI) max (94)?y2/ys 
s.t. M+ y2+y3+7 = 20 
2y1 = y2 — 9y3 
yi, y3 20 
(b) min 14y,y.y3 +100 
St. yy $2 > 3y2 — y3 
yi + 8y2 — 10 <0 
y2,¥3 = 0 


2-17 Taking the x; as variables, and all other sym- 
bols as given constants, determine whether each of 
the following is a linear or a nonlinear constraint, 
and briefly explain why. 


(a) &) 3x, + 2x. —xy7 =9 
(b) x1 +x. — 4x3 +41) <7 
(c) ike} a/x 9 + 10x13 < 100 
(d) x4/p + 11x43 > 80 
(e) BD Yj Bap? < 10 
(f) x1x2 In(@w3) < 19 
(g) [) max{x;, 3x; +2} > 111 
(h) So, fix; = 234 
2-18 Assuming that the w; are decision variables and 
all other symbols are constant, determine whether 
each of the following is a linear program (LP) or a 
nonlinear program (NLP), and briefly explain why. 
(a) Bl min 3w,+8w2-4w;3 

st. )o_, hjw; =9 


Desa OR pees) 
(b) min 5/1 Sai 16w2 
St. 6wy+wW2 > w3 
W1,W2,W3 = 0 
(c) | max ea OW; 
st. (Wi)? + wow; > 14 
W1,W2,W3< 1 
@) max ot wj/(o;)) 
st. 8 pa Wie aha Wj 
w; 2 0e P= 900 
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2-19 Determine whether a discrete or a continuous 
variable would be most appropriate to model each 
of the following quantities. 


(a) [| Amount of electricity consumed 
(b) Whether a company wins a bidding 
(c) [§] Process used to manufacture 

(d) Height of adam 


2-20 The Forest Service can build a firewatch tower 
on any of 8 mountains. Using decision variables 
x; = 1 if a tower is built on mountain j and = 0 
otherwise, write constraint(s) enforcing each of the 
following requirements. 


(a) [QJ In all 3 sites will be selected. 

(b) At least 2 of the first 5 mountains must be se- 
lected. 

(c) [| A tower should not be built on both moun- 
tains 3 and 8. 

(d) A tower can be built on mountain 4 only if one 
is built on mountain 1. 


2-21 The National Science Foundation (NSF) has 
received 4 proposals from professors to undertake 
new research in OR methods. Each proposal can 
be accepted for funding next year at the level (in 
thousands of dollars) shown in the following table 
or rejected. A total of $1 million is available for the 
year. 


Proposal 1 2 3 4 


Funding 700 400 300 600 
Score 85 70 62 93 


Scores represent the estimated value of doing each 
body of research that was assigned by NSF’s advisory 
panel. 


(a) [§] Formulate a discrete optimization model to 
decide what projects to accept to maximize total 
score within the available budget using decision 
variables x; = 1 if proposal j is selected and = 0 
otherwise. 

(b) [6] Enter and solve your model with the class 
optimization software. 


2-22 The state Department of Labor is considering 
the establishment of area job training centers at up 
to4 sites. The following table shows the land cost (in 
thousands of dollars) of the 4 sites and indicates with 
an x the sites that could provide adequate service to 
each of the five regions of the state. 


Region 1 
Northwest xX 
Southwest x 
Capital x 
Northeast x 
Southeast x 


Cost 200 40 55 


| 
alx | x | 


The Department seeks a minimum total cost collec- 
tion of sites that together could service all five re- 
gions. 


(a) Formulate a discrete optimization model to de- 
cide what sites to build using decision variables 
y; = lif site j is selected and = 0 otherwise. 


(b) [| Enter and solve your model with the class 
optimization software. 


2-23 Assuming that the z; are decision variables, de- 
termine whether each of the following mathematical 
programs is best described as a linear program (LP), 
a nonlinear program (NLP), an integer linear pro- 
gram (ILP), or aninteger nonlinear program (INLP), 
and briefly explain why. 


(a) EB] max 3z, +142. +723 

Site 10z; +5z2 + 18z3 < 25 

Z = Oo. salvia. 493 
(b) max 447; +2z. + 9823 
s.t. 12z1 + 92z2 + 8823 < 95 
320, jat...3 

(c) &) min 72422 +: 172223 + 272123 

s.t. peat a 2 


Gj 0 OTM esa tlerses eS 


(d)) min 7z, +sin(z2) + 8/z3 
st. 4z, + 16z2 + 23 < 29 
ees zs dae = dec 
(ec) Bl max 12z, 4422 
s.t. 212223 =1 
Z1,Z2 = 0 
z3 =O0orl 


(f) min 8z,-—z+14z3 
s.t. Zi = 3Z2 
Fp gh wie lisa gS 


z3 integer 


2-24 Return to the mathematical programs of Ex- 
ercise 2-23. Determine which of each of the follow- 
ing pairs of those models would normally be most 
tractable, and briefly explain why. 


(a) [$] Model (a) versus (b) 
(b) Model (b) versus (d) 
(c) [| Model (c) versus (d) 
(d) Model (b) versus (f) 


2-25 Consider the multiobjective optimization mod- 


el 
max Xj, 
max —3x, +X 
s.t. —xX; +x. <4 
xx 8 
X1,X2 = 0 
(a) Compute graphically a solution that is optimal 
if only the first objective is considered. 
(b) Compute graphically a solution that is optimal 
if only the second objective is considered. 
(c) Discuss the conflict inherent in trying to maxi- 


mize both objectives at once. 


2-26 Mexican Communications® is choosing cable 
for a new 16,000-meter telephone line. The follow- 
ing table shows the diameters available, along with 
the associated cost, resistance, and attenuation of 
each per meter. 


Diameter Cost Resistance Attenuation 
(0.1mm) ($/m)  (ohms/m) (db/m) 
4 0.092 0.279 0.00175 
5 0.112 0.160 0.00130 
6 0.141 0.120 0.00161 
9 0.420 0.065 0.00095 
1D 0.719 0.039 0.00048 


The company wishes to choose the least cost combi- 
nation of wires that will provide a new line with at 
most 1600 ohms resistance and 8.5 decibels attenua- 
tion. 


a) Formulate a mathematical programming model 
with three main constraints to choose an optimal 
combination of wires using decision variables 
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(d = 4,5, 6, 9, 12) 

xq & meters of diameter d wire used 
Assume that resistance and attenuation grow 
linearly with the length of the wire used. 


(b) [6] GEnter and solve your model with class op- 
timization software. 


2-27 The city of Lancaster’s water distribution sys- 
tem’ has 3 wells for water supply. There are 10 
pumps at these 3 wells. It is estimated that a pump- 
ing rate of 10,000 gallons per minute is needed to sat- 
isfy the city’s total water demand. There are limits 
on how much water can be pumped from each well: 
3000 gal/min from well 1; 2500 gal/min from well 2; 
7000 gal/min from well 3. There are also different 
costs of operating each pump and limits on the rate 
of each pump: 


Maximum Cost From 

Pump (gal/min) ($/gal/min) Well 
1 1100 0.05 1 
2 1100 0.05 2 
3 1100 0.05 3 
4 1500 0.07 il 
5) 1500 0.07 9) 
6 1500 0.07 3 
Th 2500 0.13 1 
8 2500 0.13 2 
9 2500 0.13 3 
10 2500 0.13 3 


Lancaster wishes to determine the least cost way to 
meet its pumping needs. 


(a) Explain why appropriate decision variables for 
a model of this problem are (j = 1,..., 10) 


x; & pump rate per minute of pump j 


Assign suitable symbolic names to the constants 
of the cost and maximum rate values in the table 
above. 

c) Formulate an objective function to minimize the 
cost of the pumping plan selected. 

Formulate a system of 3 constraints enforcing 
well capacities. 

Formulate a system of 10 constraints enforcing 
pump capacities. 

Formulate a single constraint enforcing the 
overall pumping requirement. 


®Based on L. F Hernandez and B. Khoshnevis (1992), “Optimization of Telephone Wire Gauges 
for Transmission Standards,” European Journal of Operational Research, 58, 389-392. 
7Based on S. C. Sarin and W. El Benni (1982), “Determination of Optimal Pumping Policy of a 


Municipal Water Plant,” /nterfaces, 12:2, 43-48. 
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(g) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(h) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(i) [] GEnter and solve your model with class op- 
timization software. 


2-28 A small engineering consulting firm® is estab- 
lishing its plan for the next year. The director and the 
three partners are to meet to decide which projects 
to pursue. 

Preliminary research has been done on eight 
projects. The expected profit for each project is given 
in the following table together with the number of 
person-days of background preparation each will re- 
quire and the computer processing unit (CPU) time 
(in hours) each will use. 


Project Profit Person-Days CPU 
1 2 550 200 
2 0.5 400 150 
3 3.0 300 400 
4 2.0 350 450 
5 1.0 450 300 
6 1.5 500 150 
7 0.6 350 200 
8 1.8 200 600 


Excluding downtime, it is estimated that 1000 
CPU hours will be available through the year. 
Presently there are 10 engineers (including the di- 
rector and the partners); each works 240 days per 
year. At most three engineers could be let go, and 
management does not want to hire any new engi- 
neers for next year, due to market uncertainties. A 
minimum of 3 projects need to be selected, so each 
partner will be in charge of at least one project for 
the year. The director has four favorite projects (3, 
4,5, and 8), and the company needs to select at least 
one of these. 

The firm wishes to formulate an optimization 
model to determine which projects to undertake, as- 
suming that projects must be selected on an all-or- 
nothing basis. 


(a) Justify why appropriate decision variables for 
the model are (j = 1,...,8) 
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as he if project j is selected 

oe 0 otherwise 

(b) Assign suitable symbolic names to the constants 
in the foregoing table. 

(c) Formulate an objective function to maximize to- 
tal profit from projects selected. 

(d) Formulate a pair of constraints to enforce the 
minimum and maximum engineer person-days 
available with different numbers laid off. 

(e) Formulate 3 constraints to enforce the limit on 
computer time, meet the requirement to select 
at least three projects, and include at least one 
of the director’s favorites. 

(f) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(g) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(hb) [6] Enter and solve your model with class op- 
timization software. 


2-29 A major expansion of the Brisbane airport? 
will require moving substantial quantities of earth 
from 4 sites where it is surplus to 7 locations where 
itis needed. The following table shows the haul dis- 
tances (hundreds of meters) between points, as well 
as the quantity available (m*) at each surplus site. 


Need Surplus Site 
Site Apron Term. Cargo Access 
Extension 26 28 20 26 
Dry pond 12 14 26 10 
Roads 10 ilp 20 4 
Parking 18 20 2 16 
Fire station 11 13 6 24 
Industrial park 8 10 22 14 
Perimeter road 20 22 18 21 
Quantity available 660 301 pep 99 


Quantities needed are 247 cubic meters at the ex- 
tension, 394 at the dry pond, 265 along roads, 105 
in the parking area, 90 at the fire station, 85 in the 
industrial park, and 145 along the perimeter road. 
The site engineer wishes to compute a minimum to- 
tal distance times volume plan for accomplishing the 
required earth moving. 


8Based on R. B. Gerdding and D. D. Morrison (1980), “Selecting Business Targets in a Competitive 


Environment,” Interfaces, 10:4, 34-40. 


°Based on C. Perry and M.. Iliff (1983), “From the Shadows: Earthmoving on Construction Projects,” 


Interfaces, 13:1, 79-84. 


(a) Explain why appropriate decision variables for 


a model of this problem are (i = 1,...,4, 
j=1,...,7) 
xi; & cubic meters moved from surplus 
ito need j 


(b) Assign suitable symbolic names to the constants 
of the problem. 

(c) Formulate an objective function to minimize to- 
tal distance times volume movement. 

(d) Formulate a system of 4 main constraints, assur- 
ing that the full available amount is moved from 
each surplus site. 

(e) Formulate a system of 7 main constraints, assur- 
ing that the required amount is moved to each 
needing location. 

(f) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(g) Is your model best classified as an LP, an NLP 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(h) [|] 2Enter and solve your model with class op- 
timization software. 


2-30 The state highway department would like to 
have a formula for estimating the snow removal cost 
of each snow event as a function of the number of 
inches of snow to fall. A sample of falls f; and cor- 
responding removal costs c;, j = 1,...,, has been 
collected from history. Now the department would 
like to fit these data to an S-shaped curve of the form 


k 
¢ = —— 
dee 
in a way that minimizes the sum of squared errors. 
Here k, a, and b are empirical parameters of arbi- 
trary sign. 


(a) Explain why the decision variables in this opti- 
mization problem are k, a, and b. 

b) Formulate an unconstrained optimization 

model to perform the desired curve fit. 

Is your model best classified as an LP, an NLP, 

an ILP, or an INLP, and is it single- or multiob- 

jective? 


oO 


2-31 The Blue Hills Home Corporation (BHHC)” 
employs 22 remedial education teachers to service 
special needs students at 22 schools in the St. Louis 
area. BHHC assigns its teachers to schools for an 
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entire year and is presently making decisions for the 
current year. One assignment consideration is cost; 
BHHC reimburses its teachers the cost ¢;, j of teacher 
i traveling to schoolj. However, the assignment must 
also consider 3 sets of preferences. Teachers express 
preferences scores ¢,; of teacher i being assigned to 
school j, BHHC supervisors express preferences ;;, 
and school principals provide scores p;;. In all three 
cases a higher score indicates a greater preference. 


(a) Explain why appropriate decision variables for 
a mathematical programming model of this 
problem are (i,j =1,...,22) 


1 if teacher i is assigned to 
Xij school j 
0 otherwise 


(b) Using these decision variables and the symbolic 
input constants above, express all BHHC as- 
signment goals as separate objective functions. 

(c) Write a system of 22 constraints expressing the 
requirement that each teacher be assigned to 
exactly one school. 

(d) Write a system of 22 constraints expressing the 
requirement that each school be assigned ex- 
actly one teacher. 

(e) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(fy Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


2-32 Professor Proof is trying to decide which of 6 
needed teaching assistant tasks he will assign to each 
of his 2 graduate assistants. Naturally, one assistant 
would probably be better at some tasks and the other 
assistant better at others. The following table shows 
his scoring of their potentials (high is good). 


Task 
Assistant 1 2 3 4 5 6 
0 LOOSE 285.40! 45% 0. 82) 
1 S02 70) = 90) 92°85) 480" G5: 


Professor Proof wants to assign three tasks to each 
assistant. However, tasks 5 and 6 are related and 
should be assigned to the same assistant. 


10Based on Sang Lee and M. J. Schniederjans (1983), “A Multicriteria Assignment Problem: A Goal 


Programming Approach,” Interfaces, 13:4, 75-81. 


72 Chapter Two e Deterministic Optimization Models in Operations Research 


(a) Explain why appropriate decision variables for 
an optimization model of this problem are (j = 
i140) 


0 
cou 


(b) Formulate an objective function to maximize 
the potential of the assignment chosen. (Hint: 
1 —x; =1 when x; = 0.) 

(c) Formulate a single main constraint to enforce 
requirements that at cach assistant be assigned 
three tasks. 

(d) Formulate a single main constraint to enforce 
the requirement that tasks 5 and 6 goto the same 
assistant. 

(e) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(f) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(g) [| Enter and solve your model with class op- 
timization software. 


if task j is assigned to 0 
if task j is assigned to 1 


2-33 Fast Food Fantasy (Triple-F) cooks different 
types of hamburgers j = 1,...,4 in batches. A batch 
of burger j consists of at most u; units and requires the 
entire cooking grill for 4; minutes. Assuming that the 
hourly demand for each burger is a known quantity 
d;, Triple-F would like to decide the best batch size 
for each product. All required batches (and frac- 
tions of batches) must fit within the available grill 
time each hour, and the time required to sell out 
each burger should be minimized so that none will 
get too cold waiting to be sold. 


(a) Explain why appropriate decision variables for 
a mathematical programming model of this 
problem are (j = 1,..., 4) 


x; & batch size of burger j 


(b) Formulate a system of 4 objective functions, 
minimizing the time to sell out batches of each 
burger assuming that demand is smooth over 
time. 

(c) Formulate a single constraint assuring that all 
batches needed to meet demand each hour can 
be cooked. 

(d) Complete your model with a suitable system of 
upper-bound and variable-type constraints. 


(e) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


2-34 The Kitty Railroad is in the process of planning 
relocations of freight cars among the 5 regions of the 
country to get ready for the fall harvest. The fol- 
lowing table shows the cost of moving a car between 
each pair of regions, along with the number of cars in 
each at present and the number needed for harvest 
shipping. 


Region 
From 1 2 3 4 5 
1 — 10 12 17 35 
2 10 — 18 8 46 
3 12 18 — 9 27 
4 17 8 9 = 20 
5 35 46 27 20 — 


Present 115 385 410 480 610 
Need 200 500 800 200 300 


We want to choose a reallocation plan to get the re- 

quired number of cars in each region at minimum 

total moving cost. 

(a) Briefly justify why appropriate decision vari- 
ables for this problem are (i,j = 1,...,5,i 47), 


x;; & number of cars moved from region 
i to region j 


(b) The numbers of cars x;; must physically be in- 
teger (whole numbers), but it is probably better 
to model them as continuous. Explain why. 

(c) Assign symbolic names for the constants in the 
foregoing table. 

(d) Write an objective function minimizing total 
movement cost. 

(e) Write a system of 5 main constraints, assuring 
that the net number of cars in cach region after 
the move will meet the need. 

(f) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(g) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(h) [|] BEnter and solve your model with class op- 
timization software. 


2-35 A large copper company" has 23 plants, each 
of which can burn 4 different kinds of fuels to pro- 


"Based on R. L. Bulfin and T. T. deMars (1983), “Fuel Allocation in Processing Copper Ore,” IIE 


Transactions, 15, 217-222. 


duce the energy needed in smelting. Energy require- 
ments at each plant p are known quantities r,. We 
also know the energy output ey of each ton of fuel 
f burned and the quantity of sulfur pollution sy, re- 
leased per ton of fuel f burned. Costs vary by loca- 
tion, but estimates cy, are available of the cost per 
ton for fuel f at plant p. We want to choose mixes of 
fuels at plants to fulfill energy needs while minimiz- 
ing both cost and pollution. 


(a) Briefly justify why appropriate decision 
variables for this problem are (f = 1,...,4; 
Diy se 529) 


Xp & amount of fuel f burned at plant p 


(b) Write an objective function minimizing total en- 
ergy cost. 

(c) Write an objective function minimizing total 
sulfur pollution. 

(d) Write a system of main constraints requiring 
that sufficient energy be produced at each plant. 
Also indicate how many constraints there are in 
this system. 

(e) Complete your model with an appropriate sys- 
tem of variable-type constraints. Also indicate 
how many constraints there are in this system. 

(f) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


2-36 Alabama Cabinet!” runs a sawmill produc- 
ing wood panels called “blanks” for cabinetmaking. 
Some of the wood comes from logs sawed into boards 
at the company’s mill, and the remainder derives 
from boards purchased green (undried). Lumber 
from both sources must be dried in the company’s 
kilns before being cut into blanks. 

Following are two tables, the first of which 
shows the purchase price per log, yield of green lum- 
der per log, and availability for each of the three log 
diameters. Each board foot of green lumber sawed 
from logs gives 0.09 blank. The second table shows 
the price, yield in blanks, and availability of the two 
zrades of purchased green lumber. 


LOGS PURCHASED 
Diameter $/Log Bd-ft Logs/Week 
10 70 100 50 
15 200 240 25 
20 620 400 10 
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LUMBER PURCHASES 
Grade $/bd-ft  Blanks/bd-ft Bd-ft/Week 
1 155 0.10 5000 
2; 1.30 0.08 Unlimited 


We seek a minimum cost plan for producing at least 
2350 blanks per week with the current mill’s capacity 
to saw 1500 logs and dry 26,500 board feet of lumber 
each week. 


(a) Explain why suitable decision variables for this 
model are 


Xq & number of logs of diameter d purchased 
(d = 10, 15, 20) 


ye 4 board feet of green lumber grade g 
purchased (g = 1, 2) 


(b) Numbers xz must physically be integer (whole 
numbers), but it makes sense to model them as 
continuous. Explain why. 

(c) Formulate an objective function minimizing to- 
tal material purchase cost. (We assume that 
other costs are essentially fixed.) 

(d) Formulate a main constraint assuring that the 
required number of blanks will be produced. 

(e) Formulate 2 main constraints enforcing sawing 
and drying capacities. 

(f) Formulate 4 upper-bound constraints on deci- 
sion variables. 

(g) Complete your model with a suitable system of 
variable-type constraints. 

(h) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(i) [] &Enter and solve your model with class op- 
timization software. 


2-37 The Bottles Film Festival draws thousands of 
people each year to view some of the latest motion 
pictures and award metals for the best. Planners 
are now selecting one of time slots t = 1,..., for 
each of the 7 = 1,...,m films to be shown. From 
past experience, the festival can estimate numbers 
a; 4 number of guests who would like to watch both 
film j and file j. Now they wish to schedule the films 
so that no more than 4 are shown at any time and the 
minimum total number of guests are inconvenienced 


2 Based on H. F. Carino and C. H. LeNoir (1988), “Optimizing Wood Procurement in Cabinet Man- 


siacturing,” Interfaces, 18:2, 10-19. 
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by two movies they would like to see being scheduled 
at the same hour. 


(a) Explain why appropriate decision variables for 
a mathematical programming model of this 


problemare (j= 1,.:.,m,t = 1... 7) 
1 if j occurs at ¢ 
Xt x . 
0 otherwise 


(b) Formulate an objective function to minimize 
the total number of guests inconvenienced by 
movies scheduled at the same time. (Hint: 
When x;;X,. = 1, films j and j’ are scheduled 
at the same time f.) 

(c) Formulate a system of m constraints assuring 
that each film is scheduled at some time. 

(d) Formulate a system of n constraints assuring 
that no more than 4 movies are schedule at any 
time. 

(e) Complete your model with an appropriate sys- 
tem of variable-type constraints. 

(f) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


2-38 To improve tax compliance’? the Texas 
Comptroller’s staff regularly audits at corporate 
home offices the records of out-of-state corporations 
doing business in Texas. Texas is considering the 
opening of a series of small offices near these cor- 
porate locations to reduce the travel costs now asso- 
ciated with such out-of-state audits. The following 
table shows the fixed cost (in thousands of dollars) 
of operating such offices at 5 sites i, the number of 
audits required in each of 5 states j, and the travel 
cost (in thousands of dollars) per audit performed in 
each state from a base at any of the proposed office 
sites. 


Cost to Audit 
Tax ine ee ee eee 
Site Cost 1 2 3 4 5 
1 160 0 04 #08 04 08 
2 49 Of <0 08 O04 04 
3 246 06 O04 O 05 04 
4 86 06 0:4 = 019" +0 0.4 
5 100 09 04 07 04 O 
Audits 200 100 300 100 = 200 


We seek a minimum total cost auditing plan. 
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(a) Briefly explain why appropriate decision vari- 
ables for an optimization model of this problem 
are 


x;,; 4 fraction of audits at j done from i 


1 
~A 
Vie | 0 


(b) Explain why the y; must be modeled as discrete. 

(c) Assign suitable symbolic names to the constants 
in the foregoing table: the fixed cost of office 1, 
the travel cost for audits done at j from i, and 
the number of audits at j. 

(d) Formulate an objective function minimizing the 
sum of fixed office operating cost plus travel 
costs to audit sites. (Hint: The number of audits 
done at i from j is x;; times the total number 
required at j.) 

(e) Formulate a system of 5 main constraints requir- 
ing that 100% of audits at each j be performed. 

(f) Formulate a system of 25 main constraints spec- 
ifying that no part of the audits at any j can be 
done from i unless an office is opened at i. 

(g) Complete your model with systems of variable- 
type constraints for the x and y decision vari- 
ables. 

(h) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 

(i) I] SEnter and solve your model with class op- 
timization software. 


if office i is opened 
otherwise 


2-39 The Speculators Fund is a stock mutual fund in- 
vesting in categories j = 1,..., of common stocks. 
At least a fraction €; and at most fraction u; of the 
fund’s capital is invested in any category j. The fund 
maintains estimates, v;, of the expected annual re- 
turn in capital gain and dividends for each dollar in- 
vested in category j. They also estimate the risk, rj, 
per dollar invested in each category j. The goal is to 
maximize return at minimum risk. 


(a) Explain why appropriate decision variables for 
a model of this problem are 
x; 4 fraction of fund capital invested in 


category j 


(b) Formulate an objective function maximizing ex- 
pected return per dollar invested. 


13Based on J. A. Fitzsimmons and L. A. Allen (1983), “A Warehouse Location Model Helps Texas 
Comptroller Select Out-of-State Tax Offices,” Interfaces, 13:5, 40-46. 


(c) Formulate an objective function minimizing risk 
per dollar invested assuming that risks for differ- 
ent categories are independent of one another. 

(d) Formulate a main constraint assuring that 100% 
of the fund’s capital is invested somewhere. 

(ec) Formulate a system of n constraints enforcing 
lower bounds on the fraction of fund aon in- 
vested in each category. 

(f) Formulate a system of n constraints enforcing 
upper bounds on the fraction of fund capital in- 
vested in each category. 

(g) Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


2-40 Engineers“ are designing the location for mod- 
ulesi=1,...,mfrom among the j = 1,..., 7 avail- 
able sites on a computer board. They already know 


1 ifa wire is required from 
module i to module i’ 

0 otherwise 

dj & distance between sites j and j’ 


ii! & 


Using this information, they wish to choose a combi- 
nation of locations that will minimize the total wire 
length required. 


Exercises 75 


(a) Explain why appropriate decision variables for 


a model of this problem are (i = 1,...,m; 
f= West) 
ii ul if i goes to site j 
=) 0 otherwise 


(b) Explain why the length of any wire required be- 
tween modules i and i’ can be expressed as 


n n 
aes 
j=l j'=1 
(c) Use the expression of part (b) to formulate an 
objective function minimizing total wire length. 
(d) Formulate a system of m constraints assuring 
that each module is assigned a location. 
(e) Formulate a system of n constraints assuring 
that each location gets at most one module. 
(f) Complete your model with an appropriate sys- 
tem of variable-type constraints. 
(g)-. Is your model best classified as an LP, an NLP, 
an ILP, or an INLP, and is it single- or multiob- 
jective? Explain. 


“Based on L. Steinberg (1961), “The Backboard Wiring Problem: A Placement Algorithm,” SIAM 


ew, 3, 37-50. 
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Improving Search 


To this point we have encountered a variety of types and sizes of deterministic opti- 
mization models but succeeded in analyzing only one or two. The time has come to 
begin looking seriously at solution methods. 

Some optimization models admit closed-form solutions or similarly elegant 
analysis, but the overwhelming majority are approached by numerical search— 
repeatedly trying different values of the decision variables in a systematic way until 
a satisfactory one emerges. In fact, most optimization procedures can be thought of 
aS variations on a single search theme: improving search. 

Improving search tries to better a current solution by checking others nearby. If 
any proves superior, the search advances to such a solution, and the process repeats. 
Otherwise, we stop with the current solution. Synonyms for improving search include 
local improvement, hillclimbing, local search, and neighborhood search. 

Improving search is one of the major themes of the book, and this chapter 
provides an elementary introduction. We explore the main strategies underlying im- 
proving search algorithms and identify special cases that prove particularly tractable. 
Familiarity with the model classifications of Chapter 2 is assumed. 

Readers are strongly advised to absorb thoroughly the central ideas treated 
in this chapter before proceeding to the rest of the book. Much later development 
duilds directly on Chapter 3. 


3,] IMPROVING SEARCH, LOCAL AND GLOBAL OPTIMA 


Searching means hunting, and improving searches are hunts. We trek through a 
sequence of choices for decision variables, trying to find one good enough to justify 


stopping. 
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Solutions 
Solutions are the points visited in a search. 


For example, in the Two Crude model of Section 2.1, which had decision vati- 
ables 


x1 & thousands of barrels of Saudi crude refined per day 
x2 4 thousands of barrels of Venezuelan crude refined per day 


a solution is a choice of two nonnegative quantities x; and x2. 

Notice that a solution need not be an “answer.” Any pair of nonnegative real 
numbers forms a solution in the Two Crude case, but we were satisfied in our analysis 
only by an optimal solution—one that conforms to all constraints and minimizes 
cost. 


Solutions as Vectors 


If an optimization model has n decision variables, solutions are n-dimensional. It 
is convenient to deal with them as n-vectors—linear arrays of n components. For 
example, a Two Crude example solution refining 3 thousand barrels of Saudi crude 
per day and 2 thousand barrels of Venezuelan can be expressed as the vector x = (3, 2) 
with components x, = 3 and x) = 2. 

Since improving search is about moving among such whole solutions, we use 
vector representations in most of our discussion. Primer 1 reviews vector notations 
and computations for those who feel rusty. We employ superscripts on solution 
vectors to indicate the order in which they are explored by a search. 


(a) Express the solutions in standard vector notation | 3.2]. 


(b) Identify the value of an” and ty in part (a). 


3.1 Improving Search, Local and Global Optima 


A scalar is a single real num he as 2, —0.25, 3 ee rz. Scalar variables 
such as x, v6, Ap, and a are those that take on a values. Notice that we 
always show scalar variables in italictype. 

_ Computations in operations research often involve procedures working 
on several quantities or variables at the same time. It is convenient to write 
such operations in terms of vectors—one-dimensional arrays of scalars. The 
vector may be displayed either vertically or horizontally. In this book it makes 
no difference. Representations 


are just vertical and horizontal presentations of the same vector. 

The number of scalars in a vector is its dimension. The example above is 
a 3-vector because it has dimension 3. We term the scalars making up : vector 
its components. Thus the second component of the example above is i, 

Vector variables represent vectors of scalar quantities. In this book vector 
variables are typeset in boldface (e.g., x, a, Ap), and their components are 
indicated by subscripts. Thus if p is a a (five-dimensional) variable, one 
possible value would be p = (0, —2, 2 =, 0,11) with components pz = —2 and 
ps = i. Tn this book we distinguish vectors with superscripts in ener 

£ 


a3) 


and y, , respectively. 


There are two closely related ways to conceptualize vectors geometrically. 


cheme simply thinks: of an n-vector as a point in n-dimensional space _ 
ple, 2-vectors x) = 


| (4, 1) in plot (a) in the 


( Continued) : 
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“Vectors of eo same dimension are a a subtracted col ponent by 
Spee Teta a 1) and = : 


3.1 Improving Search, Local and Global Optima 


difference which vector is mentic 
We want aes 


that es abe x 
tesult is exactly w - 


Analysis: 
(a) In the notation of | 3,2 


the four solutions are 
x = (1,0,1,2) 
x = (1,1, —2, 4) 
x® = (2,1, -1, 4) 
x® = (5,1, -1, 6) 


(b) The first component of solution 3 is oC = 5. The third component of solution 1 
() 
ISX, = —2. 


EXAMPLE 3.1; DCLUB LOCATION 


To illustrate some search ideas, consider the fictitious problem of choosing a loca- 
tion for the latest DClub discount department store. Dots on the map in Figure 3.1 
show the three population centers of the area to be served. Population center 1 has 
approximately 60,000 persons, center 2 has 20,000, and center 3 has 30,000. 

DClub wishes to locate one new store somewhere in the area in a way that 
maximizes business from the three populations. The obvious decision variables are 
x; and x, the coordinates of the chosen location. 

The new store can be located anywhere except in the congested areas within 
: mile of each population center. That is, constraints of the model are 


[a — (CDP + (2 -3)? = (2) 
( -1)°+@2-3) = ar 


(x1 — 0 + [x — (-4)P = (4) 
Figure 3.1 shades the corresponding feasible set. 
For an objective function, assume that experience shows that the business at- 
tracted from any population follows a “gravity” pattern—proportional to population 
here in thousands) and inversely proportional to 1 + the square of its distance from 
the chosen location. Using this relationship and the coordinates of the three popu- 
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ge; 


FIGURE 3.1 DClub Location Example 


lation centers, our objective function becomes 


sil pli 2g) Se ee a mean eae eek Dek 
1+ (+1)? + @ -—3)? 14+ 01-1)? + G — 3) G.) 
30 


"T+ Gy? + +4? 
Figure 3.2 provides a 3-dimensional view of the DClub example’s nonlinear 
objective function. The peak occurs near population center 1. Figure 3.3 gives an 
easier-to-read contour view (as in Section 2.2) of the full model, 


max p(x1, x2) A = = > 
P\X1, X2 ~ 14+ Ga £1? + Ge = 3) 14+ (x — 1)? + ( — 3)? 
30 


ate (patronage) 
1 2 2 
+ (x1)? + (x2 +4) (3.2) 


st. (x1 +1)? + (2 — 3)? = § (avoid 1) 
(x1 — 1)? +(@2-—3)? > 4 (avoid 2) 
(x1 — 0)? + 2 +4)? > 4 (avoid 3) 


As usual, dashed lines connect points of equal objective value. 

We want to maximize patronage p(x1, x2) subject to avoiding congested cir- 
cular areas around population centers. The point marked x“ in Figure 3.3 is (ap- 
proximately) optimal because it is the feasible point falling on the highest contour 


(principle | 2.13 |). 


3.1 Improving Search, Local and Global Optima 


Pp (1, X2) 
70 - 
60 F M \ 
50 + I , ANN 
40 + Ae 
| fi] IM \ 
FPL RA YAR - 


10 


0 


xy 5 -5 


FIGURE 3.2. Three-Dimensional View of the DClub Patronage Function 


Example of an Improving Search 


Figure 3.3 also traces an improving search leading to optimal DClub solution x“. 
Beginning at 


x = (-5,0) with pa) ~3.5 
it advances through solutions 

x = (-3,4) with = p(x) © 11.5 

x? = (-1,4.5) with p(x) ~ 21.6 

x9) — 0,3.5) with = p(x®) ~ 36.1 
to optimum 

x® = (-0.5,3) with p(x) +548 


Do not be concerned at the moment about where the various moves come from. 
Most of this chapter deals with principles for constructing such a search sequence. 

For now, simply notice why we call it an improving search. The process begins 
sith a feasible solution x© and passes exclusively through feasible points. Further- 
more, contours show that the objective function value constantly improves along the 
search path. 


3.3_| Improving searches a: 
lution to a given optimization 
points with ever-improving objective fu 
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FIGURE 3.3. DClub Example Search Leading to an Optimal Solution 


Neighborhood Perspective 

For optimization models small enough to graph, as in Figure 3.3, it is easy to spot 
whether a search path maintains feasibility and constantly improves the objective 
function. Identifying optimal solutions is not much harder. 

Unfortunately, such a global viewpoint is unavailable in typical models which 
have many decision variables. What we normally have to work with is illustrated by 
Figure 3.4. That plot zooms in on the region around x“ = (—0.5, 3) and blanks out 
the rest of the graph. We can tell something of the shape of the objective function 
near x“), and we can see the constraint limiting movement to the left. But we know 
nothing about other parts of the feasible region. 


0.8 0.7 0.6 0.5 0.4 0.3 0.2 
FIGURE 3.4 Neighborhood Perspective in DClub Search 


3.1 Improving Search, Local and Global Optima 


Such a local perspective is typical of real searches. Lacking more complete 
insight, we must base search choices mostly on information about the neighborhood 
of the current solution. 


Local Optima 

If Figure 3.4 were all we knew about our DClub model, we would be unable to 
tell whether points outside the visible range might prove superior to x“, All we 
could say is that (—0.5, 3) seems the best point in its neighborhood; that is, it is a lo- 
eal optimum. 


or focal minimum for a minimize problem) if it me feasible and if sufficienty 
small neighborhoods surrounding it contain no points that are both feasible 


and superior in objective value. 


Local Optima and Improving Search 


Observe that an improving search which has reached a local optimum can go 
no further. 


3.6 | Improving searches stop if they encounter a local optimum. 


Some solutions in the neighborhood of a local optimum may have better ob- 
jective function value [e.g., x = (—0.55, 3) in Figure 3.4]. Other neighbors may be 
feasible [e.g., x = (—0.5, 3.05)]. But no neighboring solution can continue the path of 
feasible points with ever-improving objective value (definition [3.3]) because none 
is both feasible and superior in objective value. 


Local versus Global Optima 


Truly optimal solutions to mathematical programs are feasible solutions with as good 
an objective function value as any other feasible point—ncighbor or not. To distin- 
euish this comprehensive notion of optimal, we employ the term global optimum. 
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No matter how large a radius we consider around x of Figure 3.3, for example, 
the corresponding neighborhood contains no better point. 
Unfortunately, the converse is not true. 


Figure 3.5 illustrates with another improving search of the DClub example. 


This new search conforms to definition in starting at a feasible point (in fact, 
the same one as Figure 3.3) and following an ever-improving path through feasible 
solutions. Still, it terminates at local maximum x? = (0, —3.5), where the patronage 
objective function p(x®) ~ 25.8, because no neighboring solution is both feasible 


and superior in the objective (principle [3.6]). We already know from the earlier 
search of Figure 3.3 that solution x* = (—0.5,3) has a superior objective function 


value of approximately 54.8. The strictly worst local maximum of Figure 3.5 cannot 
be globally optimal. 


\ et 

» t 

Spt? = (0.5, -2.75) 
{ 

| 

ie 


FIGURE 3.5 DClub Search Leading to a Local Maximum Not Global 


SAMPLE EXERCISE 3.2: IDENTIFYING LOCAL AND GLOBAL OPTIMA 


The figure that follows depicts constraints and contours of a minimizing optimization 
model. 


3.1 Improving Search, Local and Global Optima 


\ 
\ 
/ 
/ 
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Determine whether each of the following points is apparently a global minimum, 
local minimum, or neither. 


| (a)x® = (0,2) (b) x? = (4,5) 
(c)x® = (6,6) (d)x® = 6,0) 


Analysis: We apply definitions and é 


(a) Point x‘ is apparently a local minimum because no neighboring point has better 
objective value even though all are feasible. Still, point x® has superior objective 
function value, so x“) cannot be globally optimal. 


(b) Point x® appears to be a global and thus local minimum (principle ). No 
other feasible point can match its objective value of 20. 


(c) Even though point x has a very good objective function value, it is neither a 
local nor a global minimum because it is infeasible. 


(d) Point x is neither a local nor a global minimum because every neighborhood 


contains feasible points with lower objective function value. One example is x = 
(5.9, 0.1). 


Dealing with Local Optima 


Principle [3.6] implies that improving searches can guarantee no more than a local 
optimum because they stop whenever one is encountered. But our DClub searches 
show that local optima may not always provide the globally best solutions we pre- 
fer (principle 3.9). Must we choose between the computational convenience of 
improving search and the analytical completeness of global optima? 
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Fortunately, the answer in many of the most frequently occurring cases is “no.” 


mization models for improving search are those 
with mathematical forms assuring. Se local optimum is a global optimum. 


3.10 | The most tractable optir 


We may pursue an improving search to a local optimum knowing in advance that it 
will also provide a global. 

What can be done with the many models that fail tractability standard i 
Sometimes we can still obtain a global optimum by switching to more complicated 
forms of search. Often, we must simply settle for less. After trying several improving 
searches—typically from different starting solutions—we keep the best of the results 
as an approximate or heuristic optimum. 


available analysis i is often to run several independent: improt ving searches ad | 
accept the best local optimum discovered as a heuristi Or approxi- — 
mate optimum. _ ce oe 


3.2 SEARCH WITH IMPROVING AND FEASIBLE DIRECTIONS 


Having introduced improving search, we must now make it practical. Just how do we 
efficiently construct search paths satisfying the always feasible, constantly improving 
requirements of definition ? In this section we develop the short list of principles 
that point the way, and in Section 3.3 we translate them into algebraic conditions. 
‘Together, they comprise the foundation for nearly all practical implementations of 
improving search and much of this book. 


Direction-Step Paradigm 


Another look at the improving searches of Figures 3.3 and 3.5 will begin to reveal 
how practical search paths are constructed. Notice that the direction of search does 
not constantly change. Instead, we pursue a sequence of steps along straight-line 
move directions. Each begins at one of the numbered solutions x. There, a move 
direction is chosen along with a step size specifying how far the direction should be 
pursued. Together they determine new point xt", and the search continues. 

This direction-step paradigm lies at the heart of virtually all improving searches. 


3.2 Search with Improving and Feasible Directions 


To illustrate, consider the first move of the search in Figure 3.5, which takes us 
from x = (—5, 0) tox") = (0.5, —2.75). Most improving search algorithms would 
accomplish this move by first choosing a vector Ax of relative movement from x 
and then applying a suitable step size multiplier i. 

One vector sure to describe the direction chosen is the difference 


Ax = x) — x© = (0.5, —2.75) — (—5, 0) = (5.5, —2.75) 
A step size of A = 1 then yields the move 
x) =x 4) Ax = (—5, 0) + 1(5.5, —2.75) = (0.5, —2.75) 


However, vector Ax’ = (2, —1) defines the same direction of movement. Application 
of step size 4’ = 2.75 produces the identical move 


x) = x 4 4! Ax’ = (—5, 0) + 2.75(2, —1) = (0.5, —2.75) 


_ SAMPLE EXERCISE 3.3: DETERMINING SOI IRECTIONS AND STEP 


SIZES 

An improving search beginning at solution ee = (5,1, —1, 11) ue Os first move 
direction Aw = (0,1,1,3) for step Ay = §, then Aw® = (2,0, 3, —1) for step 
22 = 4, and finally, Aw® = (1, -i , 0,2) for step 43 = 1. Determine the solutions 
visited. 


Analysis: Applying gives 


w) = w +2, Aw? = (5,1,-1,11) + $00,113) =, $,—§, 12) 
w? = w) + A, Aw® = (5, $, —3,12) + 4(2,0, 4, -1) = (13, §, 4,8) 
w= w +3 Aw = (13,355, 8) 10, —3,0,2) = (4,1, 3,10) 


SAMPLE EXERCISE 3.4: DETERMINING MOVE DIRECTIONS FROM SOLUTIONS __ 


The first four solutions visited by an improving search are y© = (5, 11,0), y? = 
(4,9, 3), yY? = (4,9,7), and y® = (0,8,7). Determine the move directions em- 
ployed assuming that all step sizes 4 = 1. 


Analysis: With all 4 = 1, the sequence of move directions in computation 
must merely be the sequence of differences between successive solutions. First 
Ay® = y® — y® = @, 9, 3) — (5, 11, 0) = (-1, —2, 3) 
so that 
yO =y 4a Ay 
= (5, 11,0) + 1(-1, —2, 3) = @, 9, 3) 

Then 

Ay? = a ches = (4,9, 7) — (4,9, 3) = ©, 0, 4) 

Ay® = y® —y® = (0,8, 7) — (4,9, 7) = (4, -1, 0) 
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Improving Directions 


We saw in Figure 3.4 that practical improving searches usually take a local perspective, 
limiting algorithmic decisions to information about the immediate neighborhood of 
current solution x. Still, improving search definition| 3.3 |demands that every move 
improve the objective function value. 

How can we be sure of progress when we can “see” only a tiny neighborhood? 
We limit our choice to immediately improving directions. 


ion at current solution x® if the objective _ 
ior to that of x for all 1 > 0 sufficiently _ 


If a direction Ax does not immediately improve the objective in the neighbor- 
hood of current x", it will not be pursued, regardless of how it affects the objective 
over larger steps. 

Figure 3.6 illustrates for objective function (3.1) of our DClub location example. 
Constraints have been omitted since they have nothing to do with whether a direction 
improves. 

Contours in part (a) show that direction Ax = (—3, 1) improves at x® = 
(2, 0) because the (maximize) objective function increases. Notice that an improving 
direction is not required to yield progress forever. Multiple Ax = (—6, 2) (dashed 
line) remains improving even though a large enough A produces a point with an 
objective value worse than that of x®), 

Part (b) of Figure 3.6 demonstrates that an improving direction at one point 
need not improve everywhere. The same Ax = (—3, 1) that improved at x® = (2, 0) 
fails for x = (—1.5, —1.5). 


(a) Improving 


FIGURE 3.6 Improving Directions of the DClub Objective 


3.2 Search with Improving and Feasible Directions 


(b) Nonimproving 


FiGuRE 3.6 Improving Directions of the DClub Objective (Continued) 


Finally, consider local maximum x = (0, —4) in Figure 3.6(b). We can see 
from contours that x™ has a better objective function value. Still, indicated direction 


Ax’ = x? — x® = (-1,3) — 0, -4) = (-1,7) 


is not an improving direction at x". It fails definition because the improve- 
ment does not start immediately as we depart x. 


ERCISE 3. 


The following figure plots contours of a minimizing objective function over decision 
variables y; and yo. 


oF 


oO? Chapter Three e Improving Search 


Determine graphically whether each of the following directions improves at the point 
indicated. 


(a) Ay = (1, —1) aty® = 1,1) 
(b) Ay = (0, 1) aty® = (1, 1) 
(c) Ay = (0, 1000) at y? = (1, 1) 
(d) Ay = (0, 1000) at y® = (5, 3) 
(ey Ay = Ge Oaty 1G" 3) 


Analysis: We apply definition : 


(a) At y® a small movement in the indicated direction Ay = (1, —1) increases 
(degrades) the objective function value. Thus the direction is not improving. 


(b) At the same y™ a small movement in the y2-coordinate direction Ay = (0, 1) 
decreases (improves) the objective function value. Thus that direction improves at 


y, 
(c) The length of a move direction has no impact on whether it improves because 


definition addresses only sufficiently small steps 4. Thus this case improves for 
the same reasons as part (b), albeit with smaller A. 


(d) This same direction of part (c) that improved at y fails to improve at y® because 
a small step in the y2-coordinate direction increases the objective value. 


(e) Even though a move in direction Ay = (—1,0) will eventually decrease the 
objective function from its value at y° = (5, 3), the progress does not start immedi- 


ately. Thus this Ay is not an improving direction at y™. 


Feasible Directions 


Moves in improving searches of constrained optimization models must both improve 
the objective function and maintain feasibility. For the latter, practical implementa- 
tions parallel the foregoing discussion of improving directions by requiring immedi- 
ately feasible directions. 


ition x if pointx’? +2 Ax 


Just as with definition | 3.13}, we evaluate directions Ax by considering only 
the immediate neighborhood of current solution x. If feasibility is maintained for 
small enough steps, the direction is feasible. Otherwise, Ax will not be considered 
as Our next search direction because we have no way to assess its impact outside the 
neighborhood. 


3.2. Search with Improving and Feasible Directions 


FIGURE 3.7 Constraints of the Two Crude Refinery 
Example 


To illustrate, consider Figure 3.7, which depicts the variable-type and inequality 
constraints that define the feasible set of the Two Crude refinery model introduced 
in Sections 2.1 and 2.2: 


min 20x; + 15x2 
s.t. 0.3x, +0.4x. >2 
0.4x, + 0.2x2 
0.2x, + 0.3%. > (OR) 
0O<x <9 
0<x%< 6 
Every direction is feasible at solution x° = (4,4) because a short move in any 
direction leads to no violation of constraints. 
Contrast with x = (7,0). There direction Ax = (0, 1) is feasible because 
a small movement in the x-coordinate direction violates no constraints. As with 
improving directions, it does not matter that too big a step will produce a violation. 


The same direction is infeasible at x® = (0.75, 6) because even a tiny A > 0 would 
lead to a violation of the bound constraint x. < 6. 


SAMPLE EXERCISE 3.6: RECOGNIZING FEASIBLE DIRECTIONS GRAPHICALLY 


The following figure shows the feasible region of a mathematical program over de- 
cision variables y; and yo. 
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y2 


Determine graphically whether each of the following directions is feasible at the 
point indicated. 


(a) Ay = (1,0) at y 
(b) Ay = (1, 0) at y®) 
(c) Ay = (0, 1) at y? 
(d) Ay = (0, 1000) at y2 


Analysis: We apply definition : 


(a) We can move from y™ in any direction without (immediately) violating con- 
straints. Thus all directions, including Ay = (1, 0), are feasible. 


(b) At y® a small step in the y,-coordinate direction Ay = (1, 0) takes us outside the 
feasible region. Thus the direction is not feasible even though a long enough step 
would restore feasibility. . 


(c) A small step from y® in the y:-coordinate direction Ay = (0,1) violates no 
constraint. Thus the direction is feasible. 
(d) The length of a move direction has no impact on whether it is feasible because def- 


inition addresses only sufficiently small steps A. Thus this direction is feasible 
for the same reason as in part (c), albeit with smaller i. 


Step Size: How Far? 
Once an improving feasible move direction has been discovered at the current solu- 
tion, how far should we follow it? That is, what step size 1 should be applied? 

We know from definitions and that we will improve the objec- 


tion and retain feasibility for at least small steps. But having an improving feasible 
direction in hand, it is natural to pursue it as long as it remains so. 


3.2 Search with Improving and Feasible Directions 


Notice that principle involves two issues: how long the direction improves 
the objective function and how long it remains feasible. We fix 2 and choose a 
new direction when either the objective function stops improving or a constraint is 
violated. 


SAMPLE EXERCISE 3.7: DETERMINING MAXIMUM STEP SIZE 


Suppose that we are searching for an optimal solution to the mathematical program 


min 10w, + 3w2 
S.t. Wy+M2 <9 
W1, W2 = 0 


For current point w“*) = (4, 5), determine the maximum step in improving feasible 
direction Aw = (—3, —8) consistent with principle : 
Analysis: Direction Aw reduces the objective function at every point because it 
decreases both decision variables, and they both have positive costs. Thus it remains 
improving for any A > 0. 

For feasibility, we first consider the main constraint. Any step A > 0 will result 
in 

(Ww, +A Aw) + (Wo +4 Aw2) = (4— 3A) + (5 — 8A) <9 


_ That is, the constraint remains satisfied. 

We conclude that the maximum step size A will be determined by feasibility in 
the nonnegativity constraints w; > 0 and wz > 0. Any dA > 3 makes negative the 
first component of the new solution: 


4 3 4 — 3r 
(20), 22 4,,(19) ait = 
w’ =w +rdw= (5 )+a( eae fe 


Similarly, any 7 > 3 will violate nonnegativity on w2. Thus the maximum step 
retaining both improvement and feasibility in the objective function is 

Soy 2 
3° gi — 8 


Search of the DClub Example 


Algorithm 3A collects principles to in a formal improving search pro- 
cedure. We can illustrate with the DClub search shown in Figure 3.5. As usual, the 
search begins at a feasible solution vector, here x = (—5, 0). Its objective value is 
p(—S, 0) * 3.5. 

Passing to step 1 of Algorithm 3A, we look for an improving feasible direction. 
The one adopted in Figure 3.5 is Ax = (2, —1). Although there are many other 


choices, this one clearly does conform to definitions and in improving 
the objective and maintaining feasibility near x. 

We must now pick a 4 value at step 3. Any movement in the direction chosen 
leaves us at a feasible point. Thus the step size will be determined by where the 
objective function quits improving. In models with many variables, some work is 
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required to find such a maximum A. Here we can proceed graphically. Contours 
show stops after a step of approximately A; = 2.75. Thus step 4’s update yields 


xD x 4 9, Ax 
= (—5, 0) +2.75(2, —1) 
(0.5, —2.75) 


_ ALGORITHM 3A; CONTINUOUS IMPROVI 


Step 0: Initialization. Choose any starting feasible solution x, and set 
solution index t < 0. 

Step 1: Local Optimum. If no improving feasible direction Ax exists at 
current solution x, stop. Under mild assumptions about the form of the 
model, point x is a local optimum. 

Step 2: Move Direction. Construct an improving feasible direction at x 
as Ax), 

Step 3: Step Size. If there is a limit on step sizes for which direction Ax“*+)) 
continues to both improve the objective function and retain feasibility, choose 
the largest such step size as A;+1. If not, stop; the model is unbounded. 

Step 4: Advance. Update 


xD o xO se Dini Ax’) 


Then, increment ¢ <— t+ 1, and return to Step 1. 


Our first iteration isnowcomplete. Incrementingt <— 1, we return to step 1. Im- 
proving feasible directions still exist. This time the search selected Ax® = (—4, —1). 
As before, the maximum appropriate step size is determined by progress in the ob- 
jective function. Picking 42 = 0.25 leads to 


x2) < x® + 4, Ax® 
= (0.5, 22, 025-4, —1) 
(—0.5, —3) 


Returning again to step 1, we begin a third iteration. Improving feasible di- 
rections still exist, and the search chose Ax = (—1, 1). Unlike previous iterations, 
feasibility is now a consideration. A step of 0.5 in that direction brings us up against 
a constraint, even though further objective progress is possible. Thus A3 = 0.5, and 


x8 < x@ 4, Ax® 
(—0.5, —3) + 0.5(1, —1) 
(0, —3.5) 


Upon still another return to step 1, no improving feasible direction is apparent. 
Algorithm 3A terminates with locally optimal solution x = (0, —3.5). 


When Improving Search Stops 


Algorithm 3A keeps going while improving feasible directions are available because 
local progress is still possible. 


3.2. Search with Improving and Feasible Directions 


Since a direction improves and maintains feasibility for the smallest of steps 2 > 0, 
every neighborhood includes a better point. 
What if the algorithm stops at a point admitting no improving feasible direction? 


In most applications the result is a local optimum (definition ). 


al: : olution admitting 
no improving feasible direction, and mild assumptions hold, the pointisalocal _ 
ti mu m. : ¥ e zy oe 


Figures 3.3 and 3.5 illustrate this typical case. Both searches lead us to points 
where no improving feasible direction is apparent. Both results are local maxima. 

The “mild assumptions” caveat appears in principle because it is possible 
to make up examples where there are no improving feasible directions at the current 
solution, yet it is not a local optimum. Figure 3.8 shows two. The w in part (a) is 
not a local minimum of the unconstrained model indicated because it is possible to 
decrease the unusually shaped objective function for arbitrarily small steps along 
the curved path shown. Still, every straight-line direction fails to improve. Thus 
Algorithm 3A would terminate at w because there are no improving directions. 


GT 
Yajit 
“yy lif tS 7 
pl iy wel 


NS S 


(a) No improving direction (b) No improving feasible direction 


FIGURE 3.8 Nonoptimal Points Admitting No Improving Feasible Directions 


Even if the objective function has a very simple form, constraints can cause the 
same sorts of anomalies. Algorithm 3A would also terminate at the point x shown 
in part (b) because no (straight-line) feasible direction leads to higher objective 
function contours. Still, there are feasible solutions arbitrarily close to x with superior 
objective values. Stopping point x cannot be a local maximum. 

Fortunately, such examples are rare for the standard models treated in this 
book. In the interest of efficiency, analysts are almost always willing to accept the 
result of an improving search as at least a local optimum. 


oF 
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Detecting Unboundedness 


Most of the time, improving searches following Algorithm 3A will terminate at step 
1 because there is no improving feasible direction. However, they can also stop in 
A-choosing step 3. 

An optimization model is unbounded a it admits feasible solutions with arbi- 
trarily good objective value (definition | 2.20 


asible direction | 
prove oF losin: 


bility, the mode 


Figure 3.9 shows such a problem and an application of Algorithm 3A starting 
at y = (1, 0). Contour lines illustrate how the objective value improves forever as 
y, becomes large. 


FIGURE 3.9 Improving Search of an Unbounded 
Model 


The first iteration depicted in Figure 3.9 reached step 3 with chosen direction 
Ay = (1,2). Progress in that direction is limited by constraints, so the step was 


fixed at the maximum A; = 2 that retains feasibility (principle | 3.15 |). On the next 


iteration, however, the chosen improving feasible direction is Ay” = (1,0). Step 
3 terminates with a conclusion of unboundedness because there is no limit on step 
sizes improving the objective and retaining feasibility in that direction. 


Ani improving search my at ao = . 0) in ihe aiedel 


min w 
st O<w, <1 


3.3. Algebraic Conditions for Improving and Feasible Directions 


(a) Explain why the model is unbounded. 


(b) Identify an improving feasible direction Aw at w which demonstrates that the 
model is unbounded. 


Analysis: 


(a) Variable w2 has no lower bound in the model. Thus we may leave w at feasible 
value 0 and decrease w» to produce arbitrarily good objective function values. 


(b) For the reasons given in part (a), arbitrarily good feasible solutions are obtained 
by leaving w; unchanged and decreasing w2. The obvious move direction to accom- 
plish this is Aw = (0, —1). There is no limit on the 4 > 0 that may be applied to this 
direction while maintaining feasibility and improving the objective function. 


3. 3 ALGEBRAIC CONDITIONS FOR IMPROVING 
AND FEASIBLE DIRECTIONS 


What distinguishes one implementation of improving search Algorithm 3A from 
another is the process employed to identify an improving feasible direction at step 
2 (or to prove that none exists). We are now ready to develop the fundamental 
algebraic conditions at the heart of nearly every such construction. 

Readers are advised to dwell on these simple conditions and to experiment 
with their own examples and plots until each idea has been absorbed completely. We 
will return to the algebra of this section many times as we develop the algorithms of 
subsequent chapters. 


Gradients 


To obtain algebraic characterizations of improving directions, we resort to a bit of 
differential calculus. If the objective function of our optimization model is smooth 
(i.e., differentiable with respect to all decision variables), simple conditions can easily 
be devised. Readers whose calculus is a bit inadequate or rusty should not panic. 
The brief synopsis in Primer 2 includes everything you will need. 

When f is a function of n-vector x 4 (x1,..., x»), it has 7 first partial deriva- 
tives. Our principal interest is the n-vector of such partial derivatives known as the 
gradient. 

X1,...,X,), denoted V f(x), i is s the vector of rpartial 
22) a lated at x. 


3.19 | The gradient of f(x) 
derivatives Vifix) A (8f/0x1, -- 


The gradient describes the shape of the objective function because each partial 
derivative of the objective function at any current solution quantifies the slope or 
rate of change per unit change in one of the coordinate directions. 

To illustrate, we can return to patronage objective function (3.1) of Section 3.1’s 
DClub example: 


60 20 
max p(x1,%2) S = == 5 5 
Wer Ce bs fee) el Op el)e + Op 3) 
30 


+ 
1+ (x1)? + (2 + 4)? 
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a but we will 
e feel for derivatives 


book we assume only an ‘eleme 
ee to compute them occasion 


s “slopes.” 
A function is th at a ‘point if its rate of change is 
unambiguous (i.e., ifthe | den rate changes). ae prototypical 


eeep of a function that is not always differentiable i is Bigs Aix Alive 0 


the +1 appropriate oe > 0. Node 
For a function f(x) of a single - 
derivative of f with respect to x by d de or Thus constant function . 
fx) 2 a has derivative f'(x) = 0 at every x becaise its value does not 
with x. 
_ Many familial functions have easly expressed derivalives G constant): 


fo a 


x ax*' | sin(ax) — a cos(ax) 
In(ax) . cos(ax) -asin(ax) 


coy a 
g(x) nx) 


For example, the deriva i 
ee 3x 3) Be ae es 


partial derivatives of fix, x%.... =e re usually a a y a Le £ 2. 
To illustrate, consider f(x;, x2,.x3) 2 (6) ey Gs). Differentiating with x a 
x3 treated as constants produces the partial derivative 0f/0x; = SQ)! 0)! (aa). 
— affax Tey ay") and af/ax3 = (a1)° 2)’. 


3.3. Algebraic Conditions for Improving and Feasible Directions 


Figure 3.10 shows its now familiar contours. Differentiating yields 


we 
) 
Vp(x1, %2) 4 ovules 
ae 
0x2 
G3) 
120(x1 + 1) 40 (x1 Eo! 1) 60(x1) 


[1+@14+1)?+@2.-3)/P [1+@1-1?+@.-3/? [1+@)?+ 02.44? 


120(x2 — 3) 40(x2 — 3) 602 + 4) 
Wc LY eilty 3) er ea De Gy = 3): ia? Gas 4)? |? 


Thus at point x = (2, 0), 
Vp(2, 0) © (—1.60, 1.45) 


Thus in the neighborhood of x = (2, 0), function p declines at the rate of about —1.60 
per unit increase in x; and grows at the rate of roughly 1.45 per unit increase in x2. Of 
course, these rates may change if we move away from (2, 0), but they provide fairly 
complete information at points in the immediate neighborhood. 

Figure 3.10 also illustrates the geometry of gradient vectors. 


3.20 Gradients show graphically as vectors perpendicular to contours of the 
objective function and point in the direction of most rapid objective value in- 


S 


. — < < 
Vp(-15, -1.5) = (1.25, -0.25 


FIGURE 3.10 Gradients of the DClub Example Objective 
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For example, at x = (2, 0) the objective increases fastest by moving at right angles to 
the contour in direction Ax = (—1.60, 1.45), which is precisely Vp(2, 0). 


Gradient Conditions for Improving Directions 


Suppose that our search of objective function f has arrived at current solution x. 

Then the change associated with a step of size J in direction Ax can be approximated 
1 

as 


objective change ~ Sy (=) (A Axj) =A(Vf(x) » Ax) 
j J 


That is, the rate of objective function change near the current x is roughly the dot 
product of Vf(x) and Ax because it is approximated by the weighted sum of rates 
of change described by partial derivatives times move components in the various 
coordinate directions. 

When dot product Vf - Ax # 0, this gradient information provides a simple 


algebraic test of whether a direction fulfills definition as improving: 


321 
Vfix) - Ax > 0. On the other hand, if Vf(x) + Ax < 0, Ax does not improve 
at xX. — 


Cases where dot product Vf(x) - Ax = 0 cannot be resolved without further infor- 
mation. 

To illustrate, consider again the patronage function of Figure 3.10. We have 
already computed the gradient at x = (2,0) as Vp(2,0) © (—1.60, 1.45). A glance 
at the figure shows that Ax = (—1, 1) is an improving direction because it leads us to 


higher contours. Confirming characterization , we have 
Vp(0, 2)-Ax © (—1.60, 1.45) -(—1, 1) = 3.05 > 0 


On the other hand, Figure 3.10 reports that evaluation of gradient expression (3.3) at 
x = (—1.5, —1.5) yields Vp(—1.5, —1.5) © (1.25, —0.25). Testing the same direction 
Ax = (—1, 1), we see that 


Vp(-1.5, —1.5)-Ax © (1.25, —0.25)+(—1, 1) = —1.50 <0 
which verifies that direction Ax = (—1, 1) does not improve at (—1.5, —1.5). 


!This Taylor series approximation is developed more formally in Section 13.3. 


3.3. Algebraic Conditions for Improving and Feasible Directions 


Either determine by an appropriate gradient test whether each of the followin 
directions is improving for the specified objective function and point or show why 
further information is required. 


(a) Aw = (1, 0, —2) for minimize f(w) 4 (w1)? + 5w2w3 at w = (2, 1, 0). 
(b) Ay = (3, —6) for maximize f(y) 4 9y; + 40y at y = (13, 2). 

c) Az = (—6, 2) for minimize f(z) A 5(z1)? — 3z1z2 + (z2)* at z = (1, 3). 
Analysis: 


(a) Computing the objective function gradient at the indicated w yields 


of 
aw 2 2(2) 4 
Vf(w) => 2 = Sw3 = 5(0) = 0 
pia Svea) ESN S(i) 5 

0w3 


so that 
Vf(w)- Aw = (4,0,5) - (1,0, -2) = -6<0 


By applying condition , we conclude that Aw does improve at w for the mini- 
mize objective. 


(b) Computing the objective function gradient at the y indicated gives 


of. 


9 
Vify) = oe = ( a3 


dy2 
so that 
Vifly):- Ay = (9, 40)- (3, —6) = —213 <0 

Thus by condition the direction does not improve for the maximize objective. 
(c) Computing partial derivatives of the objective function at the indicated z, we have 

of 

es 1021 — 323 10(1) — 3(3) 1 

af 321 + 229 —3(1) + 2) 3 

Z2 
and 

Vf(z)-Az = (1, 3)-(—6,2) =0 


and 3. 22 | are insufficient to determine 


whether this Az improves. 
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SAMPLE E EXERCISE 3.10: CONSTRUCTING IMPROVING DIRECTIONS 


Use the shadiont of each of the following objective finenone f to construct an im- 


Chapter Three e Improving Search 


Objective Function Gradients as Move Directions 


One consequence of dot product tests and | 3.22 | is that we can derive improv- 
ing directions directly from any nonzero gradient (although we will see in Chapters 
13 and 14 that gradients are not always the best directions to choose). Since the dot 
product of a nonzero gradient with itself is 


Vix) Vf) = E (ey 0 


Ox j 


we need only choose Ax = +V/f(x). 


3.23 | When objective function gradier 
ing direction for a maximize objec 
direction for minimizing f/. 


Contours confirm that both of the gradients displayed for the maximize model 
in Figure 3.10 are indeed improving directions. We can verify the first algebraically 
by choosing 


Ax = Vp(0, 2) = (—1.60, 1.45) 
then (principle [3.21]) the dot product 
Vp(0, 2): Ax = (—1.60, 1.45) - (—1.60, 1.45) = (—1.60)” + (1.45)? > 0 
establishes that Ax improves at x = (0, 2). 


FROM GRADIENTS 


proving direction at the indicated point. 
(a) Minimize f(w) A (w1)? In(w2) at w = (5, 2). 
(b) Maximize f(y) 4 4y; + 5y2 — 8y3 at y = (2, 0, 0.5). 


Analysis: 
(a) Computing the gradient at the indicated point yields 
— 2w) In(w2) 2(5) In(2) 6.93 
Vftw) & af | (w1)? = 6) ( ae 
one W2 2 - 


Since this gradient is nonzero, we may apply construction . Negative gradient 


an =v) = ( 852) 


must improve for the minimize objective. 


3.3. Algebraic Conditions for Improving and Feasible Directions 


b) Computing the gradient at the indicated point yields 


ae 4 
ayy 
of 

fly) £ ayn 
0 
of e 
dy3 


Since this gradient is nonzero, we may again apply construction | 3.23}. Gradient 
g y ag pply 


4 
=Vfly) = 5 
£8 


will improve for the maximize objective. 


Active Constraints and Feasible Directions 


Turning now to conditions for feasible directions (definition [3.14]), we focus our 
attention on constraints. Constraints define the boundary of the feasible region 
for an optimization model, so they will be the source of conditions for feasible 
directions. 
Figure 3.11’s repeat of our familiar Two Crude example demonstrates that not 
ail the constraints of a model are relevant to whether a direction Ax is feasible at a 
particular solution x. For example, at x = (7, 0) only 


x. > 0 


poses a threat to feasibility; small movements from x“ cannot violate other con- 
straints. At x® = (0.75, 6) the relevant constraints are different: 


xo <6 and 0.4%; +0.2x, > 1.5 


3.24 | Whether a direction is feasible ata solution x depends on whether it would 
lead to immediate violation of any active constraint at xX, i.e., any constraint sat- 
isfied as equality at Xx. 


Sther names for active constraints are tight constraints and binding con- 
straints. 

Equality constraints are active at every feasible point, because they are always 
satisfied as equalities. Inequalities are more complex. For example, the Two Crude 
sonstraint 


0.4x, + 0.2x2 > 1.5 
s satisfied, but not active at x = (7, 0) of Figure 3.11 because 
0.4(7) + 0.2(0) = 2.8 > 1.5 
However, the same inequality is active at x* = (2, 3.5). There 
0.4(2) + 0.2(3.5) = 1.5 
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FIGURE 3.11 Active Constraints of the Two Crude 
Example 


Consider an optimization model with main constraints 


(w, — 4)? + 3w2 —7w3 > 31 (3.4) 
2w,+w3 < 13 (3.5) 
wy +w2+w3 = 25 (3.6) 


Determine which of these constraints are active at feasible solutions 
(a) w = (8, 20,-—3) (b) w = (5, 17, 3) 
Analysis: Equality (3.6) must be active at every feasible solution. 


(a) Checking definition for inequalities (3.4) and (3.5) at w = (8, 20, —3), we 
have 


(8 — 4)” + 3(20) — 7(-3) = 97 > 31 
and 
2(8) + (-3) = 13 
Thus only (3.5) and (3.6) are active at w = (8, 20, —3). 


(b) Checking definition 3.24 | for inequalities (3.4) and (3.5) at w = (5, 17,3), we 
find that 


(5 — 4)? + 3(17) —7(3) = 31 
and 
6) Cy = 3B 


Thus all three constraints are active at w = (5, 17, 3). 


3.3. Algebraic Conditions for Improving and Feasible Directions 


Linear Constraints 


Constraints of the Two Crude example are all linear, that is, inequalities or equalities 
constraining a weighted sum of the decision variables by a right-hand-side constant 


(definition ). We may denote general forms by 


a-x4 ) ax; >b (3.7) 
j=l 

a-xA > ax; <b (3.8) 

a-xA SY ax; =b (3.9) 
j=l 


where n denotes the number of decision variables, a; is the constraint’s coefficient 
for decision variable x;, a is the vector of coefficients aj, and b is the constraint’s 
right-hand side. 

One example is the first main Two Crude constraint, 


0.3x, + 0.4x2 > 2 
In the notation of (3.7), 
n=2, a4 =03, am=04, a=(0.3,0.4), and b=2 


Conditions for Feasible Directions with Linear Constraints 


Conditions can be devised to characterize feasible directions in more complex cases 
see Section 14.4), but we focus here on linear forms (3.7) to > (3. 9). Consider the Two 
Crude model constraint 


0.4x; + 0.2x2 > 1.5 
At solution x® = (0.75, 6) the constraint is active because 
0.4(0.75) + 0.2(6) = 1.5 


A step in move direction Ax 4 A (Ax, Ax) will change the left-hand side as 
follows: 


0.4(0.75 + 0 Ax1) +0.2(6+ A Ax2) = 1.5 +A(0.4 Ax; + 0.2 Ax2) 
Feasibility limit 
1.5+A1(0.4 Ax; +0.2 Ax2) > 1.5 
can be maintained only if we restrict the sign “net change” coefficient of J by requiring 
Ya; Axj =0.4 Ax, +0.2 Ax > 0 
j=l 


This sort of analysis leads immediately to general conditions for directions 
‘casible to linear constraints: 
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onstrained op- 


To illustrate, return to Figure 3.11. At x® = (4, 4), no constraints are active, 
and every direction is feasible. 
At x®) = (0.75, 6), there are two active constraints: 
0.4x; + 0.2x2 > 1.5 


x2. <6 


If direction Ax is to be feasible, it must violate neither. Thus principle | 3.25 | yields 
one condition for an active > form and one for an active <: 


0.4 Ax, + 0.2 Ax > 0 
Ax < 0 


Consider an optimization model with linear constraints 


3w, +w3 > 26 © (3.10) 

5w1 — 2w3 < 50 (3.11) 

2w, + w2+w3 = 20 (3.12) 

w, > 0: (3.13) 

w2 = 0 (3.14) 

State all conditions that must be satisfied for Aw to be a feasible move direction at 


w = (10,0, 0). 


Analysis: When we substitute the solution w = (10,0,0), the active constraints 


are (3.11), (3.12), and (3.14). Thus, applying conditions | 3.25 
requirements for a feasible direction are 


5 Aw, —2 Aw; <0 
2 Aw, + Aw2 + Aw3 =0 
Aw, > 0 


, the corresponding 


3.4 Unimodal and Convex Model Forms Tractable for Improving Search 


NG DIRECTIONS FOR FEASIBILITY 


_ SAMPLE EXERCISE 3.13: TES 


Return to the feasible region defined by constraints (3.10) to (3.14) and determine 
whether direction Aw = (0, —1, 1) is feasible at point w = (6, 0, 8). 
Analysis: At w = (6, 0, 8), the active constraints are (3.10), (3.12), and (3.14). Thus 
required conditions for a feasible direction are 
3 Aw, + Aw3 > 0 
2 Aw; + Aw2 + Aw3 =0 
Aw2 => 0 
Direction Aw = (0, —1, 1) meets the first two conditions because 
3 Aw, + Aw3 = 3(0) + (1) = 0 
2 Aw, + Aw. + Aw3 = 2(0) + (-1) + (1) =0 
Still, the direction is not feasible because it violates the third condition, 
Aw2 = (—1) #0 


3,4. UNIMODAL AND CONVEX MODEL FORMS TRACTABLE 
FOR IMPROVING SEARCH 


Tractability in a model means convenience for analysis (definition ). Sometimes 


difficult forms cannot be avoided, but in Chapter 1 (principle ) we saw that 
modeling often involves tradeoffs. To obtain a model that is tractable enough to yield 
useful insights, we sometimes have the choice of making simplifying assumptions or 
other compromises. 

What forms are best for improving search? In this subsection we introduce the 
classic unimodal objective function, convex feasible set cases we would always prefer 
when there is any option. 


Tractability and Local Optima 


Many issues might be raised in evaluating the tractability of model forms to improving 
search, but one stands out. The models considered most tractable to improving search 


are ones where every local optimum is necessarily global (principle ). 

Section 3.1 showed why. Improving searches stop upon encountering a local 
»ptimum because no better solution exists in the immediate neighborhood (principle 
3.6 |). If the mathematical form of the model guarantees that every local optimum is 


2 global optimum, we can pursue an improving search without being much concerned 
bout stopping at less than the truly optimal solution we desire. 


Unimodal Objective Functions 


Local optima present difficulties only when they are not global (i.e., when there exist 
ther solutions with strictly superior objective function values). Such cases cannot 
sappen if every solution improvable in this way admits an improving direction under 


definition . That is precisely the case when the objective function is unimodal. 
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‘unimodal if the oe direction from 


2 26 | Ap objective function fe 
every point in its domain to ev 
is, for every x‘) and ev 
AX= = (x — x) should be improving at 0. 


The DClub objective function of Figure 3.10 illustrates a case that fails the 
“one-hump” idea implicit in the name unimodal. To demonstrate that p(x1, x2) is not 
unimodal, we must find two solutions with different objective function values such 
that the direct path from the first to the second does not satisfy the definition of an 
improving direction. The obvious choices are the local maxima labeled x = (0, —4) 
and x = (—1, 3). The straight-line move direction 

Ax = x® —x® = (-1,3) — 0, -4) = (-1,7) 
is not improving at x because the objective function declines for small steps along 
Ax. 


Figure 3.12 graphs the more tractable objective function 
min f(y, ¥2) & (1 — 2)? — (1 — 2)(y2 — 3) + 2 = 3)* +10 
This case is unimodal because the direct path from any y™ to any y® with a lower 
(better) objective value is an improving direction. 


SAMPLE EXERCISE 3.14: RECOGNIZING UNIMODAL OBJECTIVE FUNCTIONS 


Determine graphically whether each of the following 1-dimensional objective func- 
tions f(w) is unimodal if f is maximized, and whether it is unimodal if f is minimized. 


fw) f(w) 
A A 


Analysis: With a single decision variable, the only search directions are movement 
to the left and movement to the right. 


(a) This function is not unimodal for either a maximization or a minimization. We 
can demonstrate this for maximizing f(w) by picking wY = 3 andw® = 1. Objective 
value fw?) > fw), but the (leftward) move direction from w™ to w® does not 
improve immediately at w“. For minimizing f(w) we choose w\) = 2 and w® = 0. 


Again definition is violated. 


(b) Despite the fact that this f(w) is constant from w = 2 to w = 3, it is unimodal for 
amaximization. The direct move left or right toa strictly better point always produces 
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FIGURE 3.12 Example of a Unimodal Minimize Objective 


immediate improvement. If we are minimizing f(w), case (b) is not unimodal. Point 
pairs that fail definition include w = 4 and w® = 0. 


Linear Objective Functions 


Recall that a linear objective function is one expressible as a weighted sum of the 
decision variables (definition ). The general form is 


n 
fo A So ox =e-x (3.15) 
j=l 
with c; the weight applied to decision variable x;. For instance, linear objective 
FO, 09, X82 as 2X -F Xa 
has weights cy = 3.5, cg = —2, andc3 = 1. 
Models with linear objective functions are special; one reason is unimodality. 


a2) Linear objective functions are unimodal in both maximize and minimize 
optimizationmodels. 


To see why this is true, consider general form (3.15) and any two solutions x“ 
and x). 


n n 
1 (2 Seah 
Heal Eat emits aL AG. 
j=l j=l 


ilauat 
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a Ge te 0) 

= V(x). (x® ny 
[the last because the gradient of a linear function (3.15) is just its coefficient vector 
c]. 


Now we apply Section 3.3’s gradient tests and for improving direc- 
tions. In the maximize case, f(x) > f(x) implies from the algebra above that 


fx) — fx) = Vf) (ee rs 8) S50) 
This shows that Ax = (x2) — x“) improves at x as required in unimodal definition 


because gradient condition is satisfied. An identical argument holds for 


minimize linear objectives. 


Unimodal Objective Functions and Unconstrained Local Optima 


Unconstrained local optima are solutions for which no point in some surrounding 
neighborhood has a better objective function value. That is, they are local optima 
caused entirely by the objective function. An unconstrained global optimum is a 
solution yielding a better objective value than any other in the domain of the objective 
function. Point y* = (2, 3) in Figure 3.12 is both an unconstrained local minimum 
and an unconstrained global minimum. 

Unimodal objectives are tractable because they avoid unconstrained local op- 
tima that are not global. 


3.28 | If the objective function of an optimization model is unimodal, every un- 
constrained local optimum is an unconstrained global optimum. 


This valuable property follows directly from unimodal definition [3.26]. For 
any x) not an unconstrained global optimum there exists an x) with better objective 
function value. Then unimodality implies that direction (x — x) improves at x, 
which shows that x“) could not form an unconstrained local optimum (principle 
). Every unconstrained local optimum of a unimodal objective function must 
be global. 


Constraints and Local Optima 


Objective functions are not the only parts of mathematical programming models that 
can induce local optima. For example, the only unconstrained local maximum in the 
part of the DClub feasible space displayed in Figure 3.13 occurs at unconstrained 
global optimum x = (—1, 3). 

Solutions x™ = (—1.5, 3) andx™ = (1.5, 3) illustrate a new, constraint-induced 
form of local optimum. Both are (constrained) local maxima in the full model that 
do not achieve as high an objective value as global maximum x* = (—0.5, 3). The 
difficulty is not the shape of the objective function, because improving directions 
Ax = (x* — x) at x and Ax = (x* — x®) at x lead straight to the global 
optimum. The difficulty is constraints. No neighboring point at either x or x?) 
both improves the objective function and satisfies all constraints. 
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Convex Feasible Sets 
Convex feasible sets avoid such difficulties. 


The idea is illustrated by the following feasible set: 


x2 


This set is convex because the line segment between every pair of feasible points 
lies entirely within the feasible set. The segment for x = (1, 1) and x® = (2, 3) is 
shown. 
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Contrast with the DClub feasible region of Figure 3.13. That set is not convex 
because some pairs of points fail definition| 3.29]. One of the many isx = (—1.5, 3) 
and x@) = (1.5,3). The line segment joining them includes infeasible solutions in 


both of the forbidden boxes around population centers. 
A similar situation occurs with discrete feasible sets much as 


Here only the black integer points within the main constraints are feasible. Thus 
the line segment connecting any pair of feasible points passes primarily through 
noninteger points violating constraints. 


Analysis: To show that the set fails definition | 3.29}, we need to find any pair of 
feasible solutions joined by a line segment that lies partly outside the feasible region. 
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Solutions x = (0, 2) and x) = (3, 1) sketched in the figure suffice because the line 
segment between them clearly includes infeasible points. 


Algebraic Description of Line Segments 


What does it mean to have a line segment in more than two or three dimensions? 
There is an easy algebraic characterization. Since direction (x — x) is the straight- 
line move from x to x, the line segment between the two points consists exactly 
of those solutions obtained when we add a fraction of that direction to x. 


Asanexample, suppose thatx = (1,5, 0) andx? = (0, 1, 2) are both feasible. 
One end of the line segment connecting them is x with A = 0 in 3.31 |; the other 


end is x® with A = 1. In between lie all the points with 0 < 4 < 1. For instance, the 
one for A = 0.25 is 


1 0 1 0.75 
x 40.25 fa? = x) Beg AH Beri oss e| Seals Peon fe tam | we (eae 
0 2 0 0.5 


If the feasible set is to be convex, (0.75, 4, 0.5) and all other solutions along the line 
segment must satisfy all constraints. 


SAM EXERCISE 3.16: REPRESENTING LINE SEGMENTS 


Return to the line segment joining x = (0, 2) and x® = (3, 1) in Sample Exercise 
als; 


(a) Represent the line segment algebraically. 
b) Show algebraically that one point on the line segment is x = (1, 3), 
Analysis: 
(a) Applying [3.31 |, the vectors along that line segment are those representable as 


eaten) (8) [CC (ea) 


withO<A<1. 


b) To represent (1, 3), we must choose an appropriate i. If first component 3A = 1, 
then aA = i, so that 
(3a, 2 —a) = 3(4),2-G) =, 3) 
as required. 


straint 


(1)? + (Ww)? = 1 
is not convex. 
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Analysis: We must first pick a pair of feasible solutions joined by a line segment 
that passes through an infeasible point within the unit circle, say w“) = (—1, 0) and 
w = (1,0). Under [3.31 |, the line segment between these two points includes all 
vectors expressible as 


mate) (8)f(2)-CHJECE®) 


with 0 < A < 1. Thus we have only to choose a A corresponding to an infeasible 
point to complete the argument that the set is not convex. One isi = 5, which yields 
the vector w = (0, 0) in the expression above, which clearly violates the unit circle 
constraint. 


Linear Constraints and Convexity 


As with linear objective functions, linear constraints produce unusually tractable 
feasible sets. 


| Tf all constraints of an opti n model are linear (both mai 
iable-type), its feasible space is convex. — 


How can we be sure? Pick two solutions x") and x feasible in an optimization 
model with linear constraints. For each model constraint of, say, the > form (3.7), 
feasibility means 


n nt 
> ax,” >b and a >b (3.16) 
j=l j=l 


What about points along the line segment between x and x®? We know they are 


formed as in [3.31] by some step A withO <A <1. 
Choose any such A. We may add (1 — 4) times the first inequality of (3.16) to 2 
times the second, to obtain 


n n 
(1 =a) > ax? +4. Y° ax? > 1 -ayb+ab=b 
j=l j=l 
Then a bit of regrouping gives 


n n n 
1 1 2 
y ajxy —d y ajx +) y ajxy > b 
j=l j=l j=l 


or 
: () (2) 1 
(1) 
a Qj E + AQ; ae )| >b 
ja 
The last says that point x + A(x? — x) satisfies the > constraint. Since 
we could perform exactly the same computations for each of the other constraints, 


we may conclude that points along the line segment between x and x® are all 
feasible—exactly what is required for the feasible set to be convex. 


3.4 Unimodal and Convex Model Forms Tractable for Improving Search 


Establish that the set of iG W2, W3) RHigiyiiie 


19w, + 3w2 —w3 < 14 
w, >0 


is convex. 


Analysis: Pick two arbitrary feasible solutions w” and w® satisfying both con- 
straints. Then 


(ow BwOS wh dd andi w= 0 
19w?). 6 3 Sw? 14 “andy cw, 0 


We must show that any point along the line segment joining w” and w® also 
satisfies both constraints. Each such point corresponds to some A strictly between 0 


and | in representation , SO pick such a A. 
Multiplying the foregoing main inequality for w™ by (1 — A) and that of w® 
by A, we may conclude that 


(1 =a) (19? + 309 — Ww!) + a(19w?? + 3wP — w?) 
< (1-4)14+()14 
Noting that the right-hand side is just 14 and regrouping the left, it follows that 
19 [i + Aw? — wi) 5 [9% + Aw? - w9)] 
— [ws ee Nes - w9?)] <14 


As required, the point corresponding to A on the line segment satisfies the constraint. 
For the second, nonnegativity constraint, the same computation yields 


 — raw? + rw) > (1 — 20+ AO 
or 
1) (2) 1 
wi? +a (w? — wi?) = 0 


Thus the line segment point corresponding to A also satisfies constraint w; > 0. 


Convex Sets and Constraint Tractability 


What attracts us to constraints producing convex feasible sets is the same thing that 
made unimodal objective functions desirable. Since the entire line segment between 
any pair of feasible points x‘? and x lies within the feasible set, direct move Ax = 
x) — x) must always be a feasible direction at x. 
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Models for Which Local Optima Must Be Global 


We are at last ready to characterize the most tractable optimization models for im- 
proving search—those that have no local optima that are not global. There can be 
no such local optima if improving search can move directly from any solution to any 
superior one; that is, if the straight-line path joining any solution and any better one 
must always be both an improving and a feasible direction. This is precisely what 
happens when optimizing unimodal objective functions over convex feasible sets. 


global optimum. _ « — 


Optimization models satisfying these conditions can be truly well solved by 
improving search. Those that do not conform to may be far from hopeless, but 
we can expect to have to settle for local optima that may not be global. 


SAMPLE EXERCISE 3.19: RECOGNIZING MODELS WITH LOCAL OPTIMA GLOBAL 


Use principle to show that every local minimum of the following model is a 
global minimum. 


min 8y1 — 4y3 

st. yy +3yo + y3 = 10 
0<y <5 
O0<y <3 
=< yr 29 


Analysis: The objective function is unimodal because it is linear (principle ). 
Also, all constraints are linear, which makes the feasible set convex (princi- 


ple ). It follows from principle that every local minimum is a global 
minimum. 


3.5 SEARCHING FOR STARTING FEASIBLE SOLUTIONS 


Up to now, all our discussion of improving search has envisioned moving from fea- 
sible solution to better feasible solution. But what if a first feasible solution is not 
readily available? Complex optimization problems, with thousands of constraints 
and variables, often have no obviously feasible solution. In fact, an early goal of 
analysis may be to determine whether any feasible solution exists. In this section we 
introduce the two-phase and big-M strategies, which deal with this startup issue. 


Two Phase Method 


Improving search can be adapted to deal with the absence of a starting feasible 
solution, simply by using improving search to find one. More precisely, optimization 
is effected in two phases. Phase I addresses an artificial problem with easier-to- 
satisfy constraints. Starting from some feasible solution for this artificial problem, we 


3.5 Searching for Starting Feasible Solutions 


minimize how much the artificial solution violates constraints of the true problem. 
If the violation can be driven to = 0, a completely feasible solution is at hand; 
Phase II then performs a usual improving search starting from the Phase I result. 
If the (globally) optimal value in Phase I is not 4 0, constraint violations cannot be 
completely eliminated; the true problem is infeasible. Algorithm 3B formalizes this 
two-phase method. 


| Step 0: Artificial Model. Choose any convenient solution for the true model, 
and construct a corresponding Phase | model by adding (or subtracting) 
nonnegative artificial variables in each violated constraint. 

Step 1: Phase I. Assign values to artificial variables to complete a starting 
feasible solution for the artificial model. Then begin at that solution and 
perform an improving search to minimize the sum of the artificial vari- 
ables. 

Step 2: Infeasibility. If Phase | search terminated with artificial sum = 0, 
proceed Step 3; the original model is feasible. If Phase | search terminated 
with a global minimum having artificial sum > 0, stop; the original model 

! is infeasible. Otherwise, repeat Step 1 from a different starting solution. 

_ Step 3: Phase Il. Construct a starting feasible solution for the original model 
by deleting artificial components of the Phase | optimum. Then begin at that 
solution and perform an improving search to optimize the original objective 
function subject to original constraints. 


Two Crude Model Example Revisited 


The Two Crude model of Figure 3.11 provides a familiar example: 
min 20x; + 15x2 
st. 0.3%, +0.4x. >2 
0.4%, +0.2x. > 1.5 
0.2x, +0.3x. >0.5 
0<% <=9,0< x, <6 
For this tiny case it is easy to find a starting feasible solution graphically or by trial 
and error. But for typical instances we employ a more formal procedure. 
Algorithm 3B begins by picking arbitrarily some convenient values—feasible 
or not—for the true decision variables. Often, the choice is to make all decision 
variables = 0 as we will do here. 
Our choice of x; = x2. = 0 satisfies the bound constraints 
O<x,<9 and 0<x <6 
Sull, it violates all 3 main constraints. 


Artificial Variables 


We deal with constraints unsatisfied at our arbitrary starting solution by introducing 
artificial variables to absorb the infeasibility. 


9 
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Including artificial variables x3, x4, and x5 in the 3 constraints violated at x, = 
X2 = 0 produces Phase I constraints 
0.3x, + 0.4%. +4%3 >2 
0.4x, +0.2x. +x4 > 1.5 
0.2x, + 0.3%. +x5 > 0.5 
ee cp ag she pg 
X3, X4, X5 > 0 
Notice that a different artificial variable was introduced in each constraint violated. 
It was added in each case because extra left- hand side is needed to satisfy the > 
constraints at xj =x. = 0. 
Occasionally, it is appropriate to subtract the artificial. For example, if the 
model had included a constraint 
xX} —-XxX2 = —10 
the Phase I construction with artificial x¢ is 
x1 —X%. —X» = —10 


Then for x; = x2 = 0, x6 = 10 produces feasibility. Had we used a + sign on X¢, the 
corresponding solution would require x6 negative, a violation of nonnegativity. 


Phase I Models 


Artificial variables provide a way to get started in Phase I, but our goal is to drive 
out all the infeasibility, that is, to find a solution feasible with artificial variables all 
zero. This defines the Phase I objective function. 


3,36 | The Phase I objective function minimizes the sum of the artificial vari- 
ables 


Minimizing the total minimizes each artificial because all are required to be 
nonnegative. 
In our Two Crude example, the resulting Phase I model is 
min x3+%4+.x5 
st. 0.3x,; + 0.4x2 +43 > 2 
0.4%, + 0.2x2 +.x4 > 1.5 
0.2x; + 0.3%. +x5 > 0.5 
0<% <9, 0<x <6 


X3, X4, X5 > 0 


(3.17) 
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SAMPLE EXERCISE 3.20: CONSTRUCTING A PHASE I MODEL _ 


Consider the optimization model 
max 14(w, — 10)? + (w2 — 3)? + (w3 +5)* 


st. 12w1+w3 > 19 (3.18) 
4w, + w2—7w3 <10 (3.19) 
—w1 + Ww2 — 6w3 = —8 (3.20) 
W1,W2,W3 > 0 (621) 


Construct an artificial model to begin Phase I improving search with wy = w2 = 
W3 _— 0. 

Analysis: The solution w; = w2 = w3 = 0 satisfies the main constraint (3.19) and 
nonnegativity constraints (3.21). In accord with principle , an artificial variable 
w4 must be added to deal with violated constraint (3.18), and another ws must be 
subtracted (because of the negative right-hand side —8) to satisfy the violated equality 


(3.20). 
An objective function minimizing the sum of the two artificial variables (prin- 


ciple | 3.36 |) completes the Phase I model. 
min w4+Wws 


st. 12w; +w3+wg4 > 19 
4w, + w2 — 7w3 < 10 
—w) + W2 — 6w3 — Ws = —-8 


W1, W2, W3, W4, Ws = O 


Starting Artificial Solution 


Using separate artificials in each violated constraint makes it extremely easy to com- 
plete a starting feasible solution for the Phase I search. 


For example, in Two Crude model (3.17), we have decided to initiate Phase I 
search at solution x with x? = x = 0. For these values the first main constraint 
becomes 

0.3(0) + 0.4(0) + x3 > 2 

It is violated without artificial variable x3 present, but choosing xf? = = 2 isjust enough 
to provide feasibility in Phase I. Similarly, 

0.4(0) + 0.2(0) + x4 > 1.5 
demands op = 1.5, and 

0.2(0) + 0.3(0) +.x5 > 0.5 
implies that x = 0.5. The result is x = (0,0,2, 1.5, 0.5), a starting feasible 
solution for the Phase I problem. 


fps 
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[ STARTING SOLUTION 


Return to Nile artificial problem of Sample eee 3. 20. Determine a starting Phase 
I solution having w; = w2 = w3 = 0. 


Analysis: To start Phase I, we set w = = w = = 0. Then making w? = = 


achieves feasibility in the first main constraint, and eC = 8 satisfies the last. Phase 
I would start at w = (0, 0, 0, 19, 8). 


Phase I Outcomes 


How could the Phase I search end? Certainly it will not terminate with a negative 
objective function value. Artificial variables are restricted to be nonnegative, so 
their sum must be nonnegative. For the same reason, the Phase I problem cannot be 
unbounded. The objective value cannot ever fall below zero. 

Three possibilities remain. 


3,38 | If Phase 1 terminates with a solution having (Phase 1) objective function — 
value = 0, the components of the Phase I solution corresponding to original 
variables provide a feasible solution for the original model. 


| If Phase I terminates with a ‘global minimum having tease I) objective 
~ function value > 0, the original model is ye 


3.40 _| If Phase I terminates with a local minimum ‘that may not be global but 
has (Phase 1) objective function value > 0, we can conclude notliiig. Phase | 
scart should be repeated =_— anew w starting solution. 


Begin with the happiest case [3.38]. Here Phase I has been able to drive 
the sum of the artificial variables to zero. For example, in the Two Crude Phase I 
problem above, two iterations of improving search might produce the Phase I solution 
x?) = (4, 4, 0, 0, 0) with objective function value 

xp +xP +x9 =04+040=0 
The only way nonnegative numbers can sum to zero is for all of them to equal zero. 
Thus, every artificial is necessarily zero in this final Phase I solution. Since artificials 
no longer have any effect on their constraints, the values for nonartificial variables 
must be feasible in the original model. We can simply drop all artificial variables and 
proceed with Phase II. 

The starting Phase I solution will be that part of the Phase I result that involves 
variables of the real model. For example, our Two Crude Phase I solution x = 
(4, 4, 0,0, 0) has components i = ie = 4 on nonartificial variables. Phase II 
search can start from feasible solatonl x = (4,4). 
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SAMPLE EXERCISE 3.22: VERI 


Verify that w; = 2, w2 = 0, w3 = 1 is a feasible solution to the original model in 
Sample Exercise 3.20. Then construct a corresponding optimal solution to exercise’s 
Phase I model, and explain why it is optimal. 


Analysis: The solution indicated is feasible because all components are nonnegative 
and 
122) Se519 
4(2) + (0) —-70) =1 < 10 
—(2) + (0) — 6(1) = —-8 
To construct a corresponding optimum for the Phase I problem, we set artificials 
w4 = ws = 0. Full Phase I solution w = (2, 0,1, 0, 0) is feasible because artificials 
are unneeded to satisfy constraints with w; = 2, w2 = 0, and w3 = 1. It is Phase I 
optimal because the sum of nonnegative quantities is minimum when all are zero. 


Concluding Infeasibility from Phase I 


Now consider the and cases, where Phase I improving search terminates 
with a positive objective function value. .The final solution from Phase I improving 
search will probably approximate a local minimum (principle ), but it may or 
may not be global. 

If we have some way of being sure that the Phase I solution is a global optimum, 


our conclusion is clear. The original model is infeasible (principle ), because 
every solution to the Phase I model has some artificial variables at positive values. 
Their sum just cannot be driven to zero. 

To illustrate, let us modify our Two Crude model until it is infeasible. Specifi- 

cally, reverse the direction of the last main inequality so that it reads 
0.2x, + 0.3x2 +x5 < 0.5 (3.22) 
in the Phase I problem. 

Improving search on this revised Phase I problem will terminate at some solu- 
tionx® = (2.5, 0, 1.25, 0.5, 0) with objective function value 1.25+0.5+0 = 1.75 > 0. 
Since this Phase I problem has all linear constraints and a linear objective function, 
we know from Section 3.4 (principles | 3.27}, | 3.32 |, and [3.34]) that every Phase 
I local minimum is a global minimum. It follows that no feasible solution to our 
modified Phase I example can have artificial variable total less that x®’s 1.75. That 
is, artificials simply cannot all be driven out of the solution. We conclude (principle 

3.39) that the model with last constraint reversed as in (3.22) is infeasible. 

Notice how this analysis depended on our certainty that a local minimum in 
Phase I was a global minimum. When Phase I’s solution may just be a local minimum 
with positive objective function value, there might still be a solution with artificials 
all zero. Like any improving search with local outcomes possible, we would have no 
choice but to repeat the analysis from a different Phase I starting solution (principle 


3.40). 
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Suppose that a linearly oneal Setenieaien model over PAGES Z1, 2, and Z3 


is converted for Phase I by adding nonnegative artificial variables z4, z5, and z¢. For 
each of the following original model objective functions and Phase I search results 
z, indicate what we can conclude and how Algorithm 3B processing should proceed. 


(a) Original model objective: maximize 14z,; — z3; Phase I local optimum: z = 
(1, —1, 3, 0, 0, 0) 

(b) Original model objective: minimize (z,z2)* + sin(z3); Phase I local optimum: 
z= (1,2, —3, 0, 1, 1) 

Analysis: 


(a) All 3 artificial variables = 0 in the Phase I local maximum. Thus (principle }.3.38 | ) 
z© = (1, —1, 3) provides a feasible solution for the original model. Proceed to Phase 
II search, beginning at this 2. 

(b) The corresponding Phase I model has a linear objective function and linear 
constraints. Thus z is a global optimum (principles 3.27 |, 
spite the highly non-unimodal nature of the true objective auction. were 


[3.39], we may conclude that the original model is infeasible because artificial sum 
zw+25+2=0+1+1=2 > 0ina Phase I global optimum. 


Big-M Method 


Two-Phase Algorithm 3B deals with feasibility and optimality separately. Phase I 
search tests feasibility. Phase II proceeds to an optimum. 

The Big-M method combines these activities in a single search. Artificial vari- 
ables are included in constraints exactly as in Phase I of Two-Phase search. However, 
the effort to drive their total to =0 is combined with a search for an optimal solution 
to the original model. 

The key to combining feasibility and optimality considerations is a composite 
objective function. 


aa 


The Big-M method uses a large p positive vee e M to combine feasi- 


| 1g] +M (arial variable le sum) 


Infeasibility reflected in positive values of artificial variables is forced toward zero 
by adding or subtracting that infeasibility in the objective with the large penalty 
multiplier M that gives the method its name. 

We again illustrate with the Two Crude example. The big-M form of the prob- 
lem is 
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min 20x; + 15x. + M(x3 + x4 + xs) 
st. 0.3%, +0.4%. +343 >2 
0.4x, + 0.2%. +%4 >1.5 
0.2x, + 0.3x.+x5 >0.5 
0<x, <9, 0<%x<6 
Keone Newea ) 
Notice that the constraints are identical to those of Phase I model (3.17). That means 
that the x = (0, 0, 2, 1.5, 0.5) of construction | 3.37 | continues to provide a suitable 
starting solution. 

What is new is the objective function. It combines the original objective 
to minimize 20x, + 15x2 with a large positive multiple of artificial variable sum 
(x3 + X4 + X5). 

Pick, say, M = 10, 000. Then a single search of big-M version (3.23) will lead 
to an optimal solution of the original model. Any solution with positive artificial 
variable total will be highly penalized and thus not optimal in (3.23). Among those 
that have artificial total =0, (i.e., among the feasible solutions of the original model), 
one with lowest original objective value will be preferred. 


(3.23) 


SAMPLE EXERCISE 3.24: CONSTRUCTING A BIG-M MODEL 


Return to the optimization of Sample Exercise 3.20 and construct the corresponding 

Big-M model. 

Analysis: Constraints and artificial variables are exactly as in Sample Exercise 3.20. 

Including the Big-M objective of principle results in the following big-M model: 
max 14(w, — 10)? + (w2 — 3)? + (w3 +5)? — M (wa + ws) 


s.t. 12w, + w3 +wy > 19 
4w, + w2 — 7w3 =< 0 
—w,+w2—-6w3 —ws = —8 


W1,W2,W3,W4, Ws > 0 
The artificial sum is subtracted because the objective is maximized. 


Big-M Outcomes 

Possible outcomes from a Big-M search closely parallel those of the two-phase 
method detailed in principles [3.38 | to [3.40]. When Big-M search terminates, we 
may have an optimum for the original model, a proof that the original model was 


infeasible, or just confusion. 
Consider first the optimal case. 


ocally optimal solution having all ar- 
solution corresponding to original 


For example, with M = 10,000, infeasibility is so expensive that big-M search 
of model (3.23) will compute global optimum x = (2,3.5,0,0,0). Components 
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x1 = 2 and x2 = 3.5, which were the original decision variables, constitute an optimal 
solution to model of interest. 

When Big-M search terminates with artificial variables at positive values, mat- 
ters are more murky. 


nee > 0. or the mae M may not be large enough, we can conclude 
nothing. The search should be repeated with a larger M and/or a new starting 
solution. — 


As with two-phase outcomes and , we will not be able to reach any 
conclusions when big-M search stops with a positive artificial variable total unless 
we can be sure that the search has produced a global optimum. If the outcome is 
known only to be a local optimum, a feasible solution might yet be found. 

However, notice in and | 3.44 | [3.44 | that we encounter a new difficulty when 
we adopt the big-M beNonrh If artificial variables take nonzero values in the big-M 
optimum, it may only mean that we did not choose a large enough multiplier M. 
For example, solution of big-M form (3.23) with M = 1 will yield optimal solution 
x = (0, 0, 2.0, 1.5, 0.5) because the penalty of 

M(x3 +x4 +%5) = 1(2.04+1.5+0.5) 
is insufficient to force out infeasibility. Only when M is sufficiently large—something 
that depends on details of each model—does a big- M optimum with positive artificial 
variables imply infeasibility. 


SAMPLE EXERCISE 3.25: PROCESSING BIG-M OUTCOMES — _ 
Suppose that a linearly constrained optimization model over variables z1, Z2, and 
z3 is converted for big-M by including nonnegative artificial variables z4, z5, and z, 
and penalizing the original objective function (as in| 3.41) by M = 1000 times their 
sum. For each of the following original model objective functions and big-M search 
results z, indicate what we can conclude. 


(a) Original model objective: maximize 14z, — z;; big-M local optimum: z = 
(1, —1, 3, 0, 0, 0) 


(b) Original model objective: maximize z. + z3; big-M local optimum: z = 
(0, 0, 0, 1, 0, 2) 


(c) Original model objective: minimize (zz)? + sin(z3): big-M local optimum: z = 
(1, 1, 3, 0, 0, 0) 


(d) Original model objective: minimize (z,z2)? + sin(z3); big-M local optimum: z = 
(1, 2, —3, 0,1, 1) 


Analysis: 
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a) All three artificial variables = 0 in the big-M local optimum. Thus (principle 


3.42|) z = (1, —1, 3) provides a local maximum for the original model. With both 
»bjective and constraints linear, it is also globally optimal. 


b) With both objective and constraints linear, solution z is a global optimum in 
the big-M model. However, we can conclude that the original model is infeasible 


principle ) only if we know that M = 1000 imposes a large enough penalty to 
make any infeasible solution have a lower objective value than any feasible one. 


c) All three artificial variables = 0 in the big-M local optimum. Thus (principle 


3.42 


) z = (1, 1,3) provides a local optimum for the original model. The highly 


non-unimodal nature of the objective makes it impossible to tell whether it is a 


zlobal optimum. 


d) The highly non-unimodal nature of the big-M objective function makes it impos- 
sible to know whether the indicated z is a global minimum. Thus since some artificial 
variables have positive value, our only choices (principle [3.44]) are to increase M 
and/or repeat the search from a different starting point. 


EXERCISES 


3-1 In each of the following plots, determine whether 
the specified points are feasible, infeasible, local op- 
smal, and/or global optimal in the depicted math- 
ematical program over two continuous variables. 
Dashed lines indicate contours of the objective func- 
tion, and solid lines show constraints. 


a) | Maximize problem; x® = (5,0), x? = 
@=1),x" = G,3),x” = (1, 3) 


b) Minimize problem; x = (2,4), x® = (1,3), 
x® = (2,2),x = (2,1) 


3-2 Each of the following shows the sequence of di- 
rections and steps employed by an improving search 
that began at y = (2, 0,5). Compute the sequence 
of points visited by the search. 


(oe Ag CT er oe, ye 
(-1,2, 1), a2 = 5, Ay® = (0, 6, 0), A3 = § 

(b+) Ay® = (-1,4,1), 41 = 3, Ay® = (0, 3, -6), 
i= 4 Ay = G1 As= 1 


3-3 Each of the following shows the sequence of 
points visited by an improving search. Compute the 
corresponding sequence of move directions assum- 
ing that all step sizes A = 1. 
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(a) Bl w® = 0,1,, w = (4,-1,7), Ww? = 
(4, —3, 19), w® = (3, —3, 22) 

(b) ow! = 07073), wa 2.2)7), we Go: 
w) = (3, 9, 8) 


3-4 Refer to the plots of Exercise 3-1 and determine 
graphically whether the following directions appear 
to be improving at the points indicated. 


(a) [S| Ax = (-3, 3) at x of 3-1(a) 
(b) Ax = (0,1) at x® of 3-1(a) 

(c) | Ax = (-10, 1) at x® of 3-1(a) 
(d) Ax = (4,0) at x® of 3-1(b) 

(c) [] Ax = 1, -2) at x® of 3-1(b) 
(f) Ax = (—3,2) atx of 3-1(b) 


3-5 Refer to the plots of Exercise 3-1 and determine 
graphically whether the following directions appear 
to be feasible at the points indicated. 


(a) &] Ax = (—5, 5) at x of 3-1(a) 
(b) Ax = (0,1) atx® of 3-1(a) 

(c) [| Ax = (—10, 0) at x® of 3-1 (a) 
(d) Ax = (—4, 4) at x® of 3-1(b) 

(e) [| Ax = (1, 2) at x® of 3-1(b) 
(f) Ax = (-3, 2) atx of 3-1(b) 


3-6 Consider a mathematical program with con- 
straints 

x1 — 2x2 + 3x3 < 25 

X1,X2,x%3 >0 
Determine the maximum step (possibly +00) that 
preserves feasibility in the direction indicated from 
the point specified. Also indicate whether that step 
indicates that the model is unbounded, assuming that 
directions improve everywhere. 


(a) [S|] Ax = (—1, 3, —2) from x = (4, 0, 6) 
(b) Ax = (—3, —3, 9) from x = (9, 4, 6) 
(c) Ax = (1,3, 1) from x = (0, 0, 4) 
(d) Ax = (—4, 0,3) from x = (16, 2, 1) 


3-7 For each of the following combinations of ob- 
jective function, point, and direction, determine 
whether conditions and | 3.22 | show that the 
direction improves at the point, does not improve at 
the point, or that further information is required. 


(a) | max 4y, — 2y3 + ys, y = (1,0, 19, 4, 6), 
Ay = (2, —3, 4, 0, 6) 

(b) min —y2 + 8y3 + 6y4, y = (2, 0, 8, 11, —3), Ay = 
(2, 0,3, —1, 0) 


(c) © min yyy. + (1)? + 492, y = G, 1), Ay = 
(-7, 5) 
(d) min yiy2+(1)? +4y2, y = (3, —1), Ay = (-1, 1) 


(ec) &] maxo. — 5)? +624 D2, y = 4, 0, Ay = 
(<1, 2) 
(f) min(Qy, —5)?+(02+?,y = (2, 3), Ay = (8, —6) 


3-8 Construct an improving direction from the gradi- 
ent of each objective function at the point indicated. 


(a) [6] max 3w; — 2w2 + ws at w = (2,0, 5, 1) 
(b) min 4w, + w3 —5ws at w = (1, 1,7, -1) 
(c) [OS] min (w; +2)? — wiv» at w = (3, 2) 

(d) max (w1)? — 4w, + 6w2 at w = (2, 5) 


3-9 Determine which of the constraints 
(1 — 2)? + (a2 -1)? < 10 [i] 


221 22 =8 fii] 
Zz, 20 [iii] 
220 [iv] 


are active at each of the following solutions. 


(a) z= (4,0) 
(b) z=6, 2) 


3-10 Determine whether each of the directions spec- 
ified is feasible at the solution indicated to the linear 
constraints 

3y1 — 2y2 + 8y3 = 14 

6y1 — 4y2 — 1ly3 < 11 

yi, Y2,y3 = 0 


(a) Key Ay = (0,4, 1) at y = (2,0, 1) 
(b) Ay = (0, —4, 1) aty = (2,0, 1) 
(c) KI Ay = (2,0, 1) at y = (0, 1, 2) 
(d) Ay = (0, 8, —2) at y = (0, 1, 2) 


3-11 State all conditions that must be satisfied by 
a feasible direction Aw at the solution indicated to 
each of the following systems of linear constraints. 


(a) Key 2w, +3w3 = 18 
lw; + 1w2+2w3 = 14 
W1,W2,W3 = 0 
at w = (0, 2, 6) 
(b) Same constraints as part (a) at w = (9, 5, 0). 
(c) Bl 1w1+1w2 =10 
2w, —1w2 > 8 
lw, —8w2 <1 
at w = (6, 4) 
(d) Same constraints as part (c) at w = (9, 1). 


3-12 Consider the mathematical program 
max 42;+72zZ2 
s.t. 224 + Zo 9 


Q0<z,<4 

0<7<3 
a) [| Show that directions Az = (2,0) and 
Az” = (—2,4) are improving directions for 


this model at every z. 

bd) [S| Beginning at z = (0,0), execute Improv- 
ing Search Algorithm 3A on the model. Limit 
your search to the two directions of part (a), and 
continue until neither is both improving and fea- 
sible. 

Show in a two-dimensional plot the feasible 
space and objective function contours of the 
model. Then plot the path of your search in 
part (b). 


oO 


3-13 Do Exercise 3-12 for mathematical program 


min Zz; + Z2 

s.t. Z+2mz>4 
O0<ma <5 
0<z <3 


directions Az = (0, —5), Az® = (—2, 1), and ini- 
sal point 2 = (5, 3). 

3-14 Determine whether each of the following objec- 
tive functions is unimodal over the domain specified, 
and for those that are not, indicate points x and x 


that violate definition 3.26 s 


2) [&] The following maximize objective over x in 
the domain shown. 


f() 


>) The following minimize objective over x in the 
domain shown. 
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f(x) 


(c) EQ] The following maximize objective over 
(x1, X2) in the domain shown. 


(d) The function of (c) if it is being minimized. 

(ec) |] The maximize objective function plotted 
in Exercise 3-1(a) over (x1, x2) in the domain 
shown. 

(f) The minimize objective function plotted in Ex- 
ercise 3-1(b) over (x;, x2) in the domain shown. 

(g) [| min 3x; — 17x. + 7xs over all (x;,...,X5). 


(h) max 37x, — 100x. — 13x3 + 40x4 over all 
(irik opts) 


3-15 Consider the line segment between each of the 
following solution pairs z and z®. Write an alge- 
braic expression representing all points on the line 
segment, show that the given z®) is on the line seg- 
ment, and show that the z specified is not. 


(a) Bz = @G,1,0), 2 = (0,4,9), 2 = 
(2, 2, 3),z = (3, 5,9) 

(b) z = 2,1,4), 2 = (7,0, -1), 2 = ©, §,0), 
z® = (0, 3,1) 
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3-16 Determine whether the feasible set for each of 
the following systems of constraints is convex, and if 
not, indicate points x and x® that violate definition 


[529], 
(a) BI 


(x1)? + (x2)? = 9 
xy +x < 10 


X2,X2 = 0 


(b) (41)? + (2)? < 25 
2x1 + 3x2 > 6 
X1,xX2 >0 


(c) ] x, — 2x. +x3 =2 
x1 + 8x2 —x3 < 16 
x1 + 4x2 -43>5 
X1,%X2,%3 > 0 
@) d.,4; = 1000 
0 <x, < 100, j=1,...,20 
(e) KB] x, + 2x2 + 3x3 +x4 < 24 
0<x <10,j=1,...,4 
x; integer, j=1,...,4 


(f) 2x; +x. +x3 —5x4>3 
Gj = Oona) = ee, 4 


3-17 Construct a Phase I model corresponding to 
each of the following, and indicate appropriate start- 
ing values for the artificial variables. Assume that all 
original decision variables start at w; = 0. 


(a) &] max 22w, — w2 +1503 
s.t. 40w, + 30w2 + 10w3 = 150 
WwW, —W2 <0 
4w2 + w3 > 10 
W1,W2, W3 > 0 


SUGGESTED READING 


Bazarra, Mokhtar S., Hanif D. Sherali, and C. M. 
Shetty (1993), Nonlinear Programming Theory and 
Algorithms, Wiley, New York. 


(b) min —11w; +10w2 + w3 
st. 3w, +w2+2w3 <9 

4w, + 4w2 > 6 
Wy —W2 —w3 = —2 

W1,W2,W3 = 0 


(c) KB] min 2w, +3w2 
Si (Ww, — 3) + (W2 - 3)? <4 
2w, +2w2 =5 
W= 3 


(d) max ww 
st. (w, — 5)? + (wn)? < 25 
3w1 -w>= 0) 
W1= 2, 
W2 => 0 


3-18 Describe how a two-phase improving search of 
a model with original variables y;, y2, and y3 would 
proceed if Phase I search terminated as follows: 


(a) [8] Global optimum y = (40, 7, 0, 9, 0) 

(b) Global optimum y = (22, 4, 3, 0, 0) 

(c) [Q] Local optimum y = (1, 3, 1, 0, 0) 

(d) Localoptimum y = (0, 2, 6, 5, 1), which may not 
be a global optimum 


3-19 Construct a Big-M starting model for each case 
in Exercise 3-17 and indicate appropriate starting 
values for the artificial variables. Assume that all 
original decision variables start at w; = 0. 

3-20 Describe how Big-M search of a model with 
original variables y;, yo, and y3 would proceed if im- 
proving search of the Big-M version produced each 
of the outcomes in Exercise 3-18. 


Luenberger, David G. (1984), Linear and Nonlinear 
Programming, Addison-Wesley, Reading, Mass. 


( H A P T £ R 


Linear Programming 
Models 


Linear programs (or LP’s) are mathematical programs combining all the character- 
istics that Chapter 3 showed lead to high tractability. 


| An optimization model is a linear program (or LP) if it has continuous 
ables, a single linear objective function, and all constraints are linear equal- 
ities or inequalities, 


That is, the model’s single-objective function and all constraints must consist of 
weighted sums of continuous decision variables. 

Everything in life is not linear and continuous, but an enormous variety of 
applications can be validly modeled as LPs. Cases with thousands, millions, or even 
billions of variables can then be solved to global optimality. 

We have already encountered linear programming models in Sections 2.1 to 
2.4. Many others are found in network flow Chapter 10. 

In this chapter we develop a more comprehensive picture by presenting several 
of the classic LP application forms. Most of the models described were actually used 
by real organizations. Only the details and numerical data are fictitious. 

Linear programs are so central to the study of mathematical programming 
that four subsequent chapters also treat LP topics. In Chapter 5 we specialize the 
improving search notions of Chapter 3 to obtain a powerful algorithm called the 
simplex method; in Chapter 6 we present newer, interior point varieties of improving 
search for LPs; in Chapter 7 we develop the extensive duality and sensitivity analysis 
available in linear programming; and in Chapter 10 we address the special linear 
programs that model network flows. 


4,1 ALLOCATION MODELS 


One of the simplest forms of linear programs that occurs widely in application might 
be termed allocation models. The main issue is how to divide or allocate a valuable 
resource among competing needs. The resource may be land, capital, time, fuel, 
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or anything else of limited availability. For example, the (nonlinear) E-mart model 
(2.13) of Section 2.4 allocated an advertising budget. 


EXAMPLE 4.1: FOREST SERVICE ALLOCATION 


The US. Forest Service has used just such an allocation model to address the sensi- 
tive task of managing 191 million acres of national forestland.'! The Forest Service 
must tradeoff timber, grazing, recreational, environmental, national preservation, 
and other demands on forestland. 

Models of a forest begin by dividing land into homogeneous analysis areas. 
Several prescriptions or land management policies are then proposed and evaluated 
for each. The optimization seeks the best possible allocation of land in the analysis 
areas to particular prescriptions, subject to forest-wide restrictions on land use. 

Table 4.1 provides details of the fictional, 788 thousand acre Wagonho National 
Forest that we model. Wagonho is assumed to have 7 analysis areas, each subject 
to 3 different prescriptions. The first prescription encourages timbering, the second 
emphasizes grazing, and the third preserves the land as wilderness. Using index 
dimensions 


i A analysis area number (i = 1,..., 7) 
j A prescription number (j = 1,..., 3) 
Table 4.1 provides values for all the following symbolic parameters: 


s; 4 size of analysis area i (in thousands of acres) 


pi & net present value (NPV) per acre of all uses in area 

iif managed under prescription j 
t,; & projected timber yield (in board feet per acre) of analysis area 

iif managed under prescription j 

gi; & projected grazing capability (in animal unit months per acre) of analysis 
area i if managed under prescription j 

wi; & wilderness index rating (0 to 100) of analysis area i if managed under 
prescription j 


We wish to find an allocation that maximizes net present value, while producing 
40 million board feet of timber, 5 thousand animal unit months of grazing, and 
keeping average wilderness index at least 70. 


Allocation Decision Variables 


As in all such models, our Forest Service example seeks an optimal allocation of a 
valuable resource. Corresponding decision variables define the allocation. 


‘how much of the 


‘Based on B. Kent, B. B. Bare, R. C. Field, and G. A. Bradley (1991), “Natural Resource Land 
Management Planning Using Large-Scale Linear Programs: The USDA Forest Service Experience with 
FORPLAN,” Operations Research, 39, 13-27. 
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TABLE 4.1 Forest Service Example Data 


Analysis Acres, Prescrip- NPV, Timber, Grazing, Wilderness 
Area, S; tion, (per acre) (peracre) (per acre) Index, 
i (000)’s Jj Piy Gi 8; Wij 
1 HS) 1 503 310 0.01 40 
2 140 50 0.04 80 
3 203 0 0 95 
2, 90 1 675 198 0.03 55 
2 100 46 0.06 60 
B 45 0 0 65 
3 140 1 630 210 0.04 45 
2; 105 57 0.07 55 
3 40 0 0 60. 
4 60 1 330 112 0.01 30 
2 40 30 0.02 35 
3 295 0 0 90 
5 2D) 1 105 40 0.05 60 
2 460 32 0.08 60 
3 120 0 0 70 
6 98 1 490 105 0.02 35 
Dats 55 25 0.03 50 
3 180 0 0 75 
7 113 il 705 213 0.02 40 
2 60 40 0.04 45 
3 400 0 0 95 


Our Forest Service case will employ nonnegative 


x;,; 4 number of thousands of acres in analysis area i managed by prescription j 


Forest Service Allocation Model 


The Forest Service’s objective is to maximize total net present value (NPV). In terms 
»f the defined notation, this is 


max 5: 5 PijXij = 503X1,1 + 140x712 + 203x1,3 + 675x2,1 + 100x2,2 + 45x2,3 
lati + +--+ 705x7,1 + 60x7,2 + 400x7,3 
One system of constraints must assure that all acres of each analysis area are 
allocated. For example, in analysis area 1, 
X11 +%1,2 +%1,3 = 75 


Using symbolic constants, all 7 such constraints can be expressed by the system 


3 
tas ee 
jt 


Finally, there are the performance requirements on timber, grazing, and wilder- 
sess index. For example, we want timber output 
are 
So DS tipxig = 310211 + 50x12 + 01,3 + 198%2,1 + 46x22 + O23 
i=1 j=l 
saat 213x71 + 40x7,2 + 0x73 
> 40, 000 
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Combining produces the following allocation linear program: 


ia) 
max) ) | Pixs 
i=l j=l 


(present value) 


3 
s.t. yi Nias easel hee (allocation) 
t,jX;j; = 40,000 (timber) 
=a (4.1) 
ALi 3 
Ss Ds, Cut eS (grazing) 
i=1 j=l 
fica 
== > aN Wi jXij; = 70 (wilderness) 
ie ere ia 
j 20 a leerne/ae sealer tO) 
An optimal allocation aren 
x Xi = 09, X73 = 75, X35, =90, x3,=0, x3,=0 
x3,= 140, x3,=0, 1x35,=0, 4x4,=0, 2x7,=0, x7, =60 
x5, = 0, x55 = 154, x5, =58, x16, =0, x6,=0, 16, = 98 
x7, = 0, x7 = 0, x73 = 113 


with total net present value $322,515,000. 


‘SAMPLE EXERCISE 4.1: FORMULATING ALLOCATION LPS 


Jill College is taking courses in operations research, engineering economics, statis- 
tics, and material science. She has 30 study hours to prepare for her finals and wishes 
to divide her time to improve her term grades as much as possible. Naturally, her 
favorite course is operations research, so she will spend as much on it as any other. 
Still, she believes up to 10 hours of study could be useful in any of the courses, with 
each hour on operations research increasing her grade by 2%, each on engineer- 
ing economics yielding 3%, each on statistics producing 1%, and each on materials 
science adding 5%. Form an allocation linear program to help Jill optimize her study. 


Modeling: Using decision variables 


the required model is 


max 
s.t. 


h; & hours spent on the jth course 


2h, + 3h2 + 1h3 + Shy 
hy +ho +h3+hyg = 30 


h=h (pS pane! 
helo yok. 4 
hy = 0 = eee! 


(total gain) 
(allocation) 
(OR most) 
(maximum 10) 


The objective maximizes score gain. Main constraints make the allocation total 30. 
keep OR study as great as any other, and limit the allocation for any class to 10 hours. 


4.2 Blending Models 


4,2 BLENDING MODELS 


Allocation models split a resource. Blending models combine them. Thatis, blending 
models decide what mix of ingredients best fulfills specified output requirements. 
Various applications blend everything from chemicals, to diets, to metals, to animal 
food. For example, the Two Crude case of Section 2.1 is a blending model mixing 
crude petroleums to produce refinery products. 


EXAMPLE 4.2: SWEDISH STEEL 


The steel industry confronts another blending problem when it melts materials in 
high-temperature furnaces to manufacture new alloys from scrap. Fagersta AB of 
Fagersta, Sweden, is one of many companies that have used mathematical program- 
ming to plan this steel blending. 

An optimization arises each time a furnace is charged. Scrap in the available 
inventory is combined with pure additives to produce a blend having the required 
percentages of various chemical elements. It is critical to make maximum use of 
scrap because additives are much more expensive. Although there are some integer 
programming aspects discussed in Section 11.1, we deal here only with the simpler 
linear programming form. 

Our fictitious version of Swedish steelmaking will produce a 1000-kilogram 
furnace charge. All steel consists primarily of iron. Table 4.2 shows the much smaller 
fractions of carbon, nickel, chromium, and molybdenum in the four available supplies 
of scrap, on which we can draw, along with the quantities held and their unit cost in 
Swedish kroner. It also shows the three higher-cost additives that can be used and 
the acceptable ranges for the resulting blend. For example, the 1000 kilograms of 
steel produced should contain between 0.65 and 0.75% carbon. 


TABLE4.2 Data for Swedish Steel Example 


Available Cost 


Composition (%) (kg) (kr/kg) 
Carbon Nickel Chromium Molybdenum 

First scrap 0.80 18 12 — 13 16 
Second scrap 0.70 BD. 11 0.1 250 10 
Third scrap 0.85 ram =e a Unlimited 8 
Fourth scrap 0.40 = = — Unlimited 9 
Nickel es 100 ae —_ Unlimited 48 
Chromium ea ES 100 — Unlimited 60 
Molybdenum r, = = 100 Unlimited 53 
Minimum blend 0.65 3.0 1.0 algal 

Maximum blend 0.75 39 12. ales} 


Ingredient Decision Variables 


It is characteristic that we must make ingredient decisions. 


Based on C.-H. Westerberg, B. Bjorklund, and E. Hultman (1977), “An Application of Mixed 
Integer Programming in a Swedish Steel Mill,” Interfaces, 7:2, 39-43. 
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A " much of each 


In our Swedish Steel example we have seven such variables: 
x; 4 number of kilograms of ingredient j included in the charge 
where j = 1,...,4 refers to the four supplies of scrap, andj = 5,...,7 to the pure 
additives. 
Composition Constraints 


One requirement on any solution to our example is that the total charge sum to 1000 
kilograms: 


Xy bx. +.x3 +2x4 +45 + X6 +7 = 1000 (4.2) 
However, most of the main constraints restrict the composition of the mix. 


rosit ¢ upper and/or lower _ 
_ limits o on n the properties of the resulting blend. | 


In our Swedish Steel example we have both upper and lower limits on the frac- 
tion of carbon, nickel, chromium, and molybdenum in the mix. Each such constraint 
will have the form 


fraction in sore of > /allowed bbe 

Y ith P ion i ; 4, 

j ; ingredient or | fraction in Gate ) (4.3) 
j \ ingredient aod the blend 


Specifically, the composition constraints required are 


Vv 


lA 


7 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 = 0.0065 ‘i Xj 
j=l 


7 
0.0080x; + 0.0070x2 + 0.00853 + 0.0040x4 < 0.0075 — Xj 


0.180x1 + 0.032x2 + 1.0x5 > 0.030 5° x; 
j=l 
. 
0.180x1 + 0.032x2 + 1.0xs < 0.0859) xj 
j=) 
7 
0.120x1 + 0.011x2 + 1.0x¢ ert) (ee Sie 
j=1 
: z 
0.120x1 + 0.0112 + 1.0x6 < 0.012 D> x; 
j=1 
- 
0.001x2 + 1.0x7 > 0.011 5° x; 
j=1 
- 
0.001x2 + 1.0x7 20.013; 5%) 


4.2 Blending Models 


4 food blending model with 3 se HAO employs Hhiinebative aeaaion variables 
x; & grams of ingredient j used 


where ingredient 1 is 4% fiber and has 10 milligrams (mg) of sodium per gram, 
agredient 2 is 9% fiber and has 15 mg of sodium per gram, and ingredient 3 is 3% 
aber and has 5 mg of sodium per gram. Formulate linear constraints enforcing each 
of the following requirements. 


a) The blend must average at least 5% fiber. 
5) The blend must contain at most 100 mg of sodium. 
Vodeling: 
a) Following format (4.3), the needed constraint is 
0.04x, + 0.09x2 + 0.03x3 > 0.05(x1 + x2 + x3) 

5) Here the constraint deals with absolute amounts, not fractions. The required 
form is 

10x, + 15x2 + 5x3 < 100 


Swedish Steel Example Model 


Collecting the elements derived so far yields the following LP model of our Swedish 
Steel example: 


min 16x; + 10x2 + 8x3 + 9x4 + 48x5 + 60x65 + 53x7 (cost) 
St. Xp $X24+x34+X4+X5+2X6 +x, = 1000 (weight) 
0.0080x; + 0.0070x2 + 0.0085x3 (carbon) 


+0.0040x4 > 0.0065 (1000) 
0.0080x1 + 0.0070x2 + 0.0085x3 
+0.0040x4 < 0.0075(1000) 


0.180x; + 0.032x2 + 1.0x5 > 0.030(1000) (nickel) 

0.180x,; + 0.032x + 1.0x5 < 0.035(1000) (4.4) 
0.120x, + 0.011x2 + 1.0x¢6 > 0.010(1000) (chromium) 

0.120x; + 0.011x2 + 1.0x¢6 < 0.012(1000) 

0.001x2 + 1.0x7 > 0.011(1000) (molybdenum) 
0.001x2 + 1.0x7 < 0.013(1000) 

Se] 7) (available) 

x2 < 250 

Bolin caseXa oe) (nonnegative) 


The objective function merely sums the costs of the ingredients used. The only 
constraints not discussed above enforce supply limits on the first two types of scrap. 
-omposition constraints have been simplified by taking advantage of the fact that 
‘otal weight is fixed at 1000 kilograms. 
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The unique optimal solution to model (4.4) has 
x4 = 75.00kg, x3 =90.91kg, x3 = 672 23 kept =e lo] ks 
x¥=13.59kg, x§ =0.00kg, x7= 10.91 kg 
The total cost of an optimal charge is 9953.7 kroner. 


Ratio Constraints 


The composition constraints of our Swedish Steel example can be viewed as ratio 
constraints because they bound the fraction that one weighted sum of variables forms 
of another. For example, the lower limit on carbon is 


7 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 > 0.0065 3 a 
j=l 
or 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x.4 
XX. + x3 +.X4 + x5 + Xo + X7 
In the latter form, the constraint does not even appear linear. Still, multiplying 
by the denominator, which we know will be nonnegative in every feasible solution, 
produces the linear version without reversing the direction of the inequality. 
Many blending models have similar ratio constraints among ingredients. For 
example, if ingredients 3 and 4 must be in ratio no more than 2:3, we have the 
requirement that 


> 0.0065 


x3 2 
—_— = _ 

X4 3 

Again, since we know the sign of denominator x4 = 0, we may cross-multiply to 
produce the linear form 


2, 
x3 5 3X4 


Contrast this with the ratio constraint 
X3 


2 
< = 
Pilea.) 3 
Over nonnegative x; the denominator of the latter constraint has an unpredictable 
sign. Without further information it must be considered nonlinear. 


ound the 
| to linear 


nonnegative decision variables x1, x2, and x3 determining the amount of three sub- 
stances in a blend. 
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(a) The amounts of substance 1 and 2 should be in the ratio 4:7. 
(b) The amount of substance 1 is at most half that of substance 3. 


(c) The blend is at least 40% substance 1. 
Modeling: 


(a) The constraint required is 


(b) The constraint needed is 


(c) The constraint specified is 
x4 


———— > 0.40 
xX) +X. +x3 


or (with the nonnegative denominator) 


x1 = 0.40%, + x2 + x3) 


4,3, OPERATIONS PLANNING MODELS 


Another classic linear program form deals with operations planning. In organizations 
ranging from volunteer, to government, to manufacturing, to distribution, planners 
must decide what to do and when and where to do it. For example, Section 2.3’s 
Pi Hybrids model (2.10) was used to plan the production and distribution of seed 
corn. 


EXAMPLE 4.3: TUBULAR PRODUCTS OPERATIONS PLANNING 


Sometimes a plan involves nothing more than allocation of work to operations. The 
tubular Products Division (TP) of Babcock and Wilcox encountered just sucha prob- 
©m in investigating how work should be reallocated upon opening a new mill. TP 
manufactured steel tubing in a variety of sizes and for many different uses, including 
<lectrical power generation. At the time of the study three mills handled production. 
The object was to consider how a fourth mill of different configuration would affect 
‘he optimal distribution of work (and associated costs) among the mills. 

Table 4.3 shows fictional data for existing mills 1 to 3 and one design for new mill 
+. versus an array of 16 products. The products comprise all combinations of standard 
or high-pressure tubing; 5° 1-, 2-, or 8-inch diameters; and thick or thin tube walls. 
the table includes the cost (in dollars) per 1000 pounds of each product according 
2 which mill does the work, and the required processing time (in hours) per 1000 
pounds produced. Missing values indicate products that cannot be manufactured 
‘casibly at the mill indicated. 


3Based on Wayne Drayer and Steve Seabury (1975), “Facilities Expansion Model,” Interfaces, 5:2, 
vert 2, 104-109. 
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Table 4.3 also shows the assumed division-wide demand for each of the 16 products 
in thousands of pounds per week. At present the three existing mills 1 to 3 have 800, 
480, and 1280 hours per week of effective production capacity, respectively. New 
mill 4 is planned for 960 hours per week. 


TABLE 4.3 Tubular Products Example Data 


Mill 1 Mill 2 Mill 3 Mill 4 
I Weekly 
Cost, Hours, Cost, Hours, Cost, Hours, Cost, Hours, Demane 
Product Cpt tl Cp,2 by,2 Cy,3 by3 Cp,4 by4 d, 
Standard 
1: } in. thick 90 0.8 6) 0.7 70 0.5 63 0.6 100 
2 3 in. thin 80 0.8 70 0.7 65 0.5 60 0.6 630 
5 lin. thick 104 0.8 85 0.7 83 0.5 77 0.6 500 
4: 1in. thin 98 0.8 79 0.7 80 0.5 74 0.6 980 
5:5 2anathick 123 0.8 101 0.7 110 0.5 99 0.6 720 
6: 2in. thin 113 0.8 94 0.7 100 0.5 84 0.6 240 
a 8 in. thick — — 160 0.9 156 0.5 140 0.6 75 
8: 8 in. thin — — 142 0.9 150 0.5 130 0.6 22 
Pressure 
9: 3 in. thick 140 1.5 110 0.9 — — 122 1.2 50 
10: 5 in. thin 124 15 96 0.9 — — 101 2) 22 
11:  lin.thick 160 ules) 133 0.9 —_— _ 138 1.2 353 
12: 1in. thin 143 1.5 127 0.9 — — 133 1.2 55 
13: <2insthick  -202 Iles} 150 0.9 — — 160 ulee 125 
14: 2 in. thin 190 TS 141 0.9 — — 140 need 35 
15: 8 in. thick o — 190 1.0 — — 220 eS) 100 
16: 8 in. thin —_— -— 175 1.0 —_ — 200 ales} 10 


Tubular Products Operations Planning Model 


In operations planning models, the decisions always revolve around what operations 
to undertake. Here the problem has two index dimensions: 


p & product number (p = 1,..., 16) 
m 4 mill number (m =1,..., 4) 
The corresponding decision variables are 


Xp,m & amount of product p produced at mill m 
(in thousands of pounds per week) 


In terms of these decision variables, it is straightforward to produce a linear 
programming model. For brevity, define the symbolic constants 


Cp,m & unit cost of producing product p at mill m shown in Table 4.3 [= +00 if this 
(p,m) combination is impossible] 


tp,m & unit time to manufacture product p at mill m shown in Table 4.3 [= 0 if this 
(p,m) combination is impossible] 


d, & weekly demand for product p shown in Table 4.3 


bm & given production capacity at mill m 
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Then the model required is 
16 4 


min a a Cp,mXp,m (total cost) 
p=1 m=1 
4 
s.t. > Xp,m = dp Di Ne 6 (demands) 4.5) 
m=1 ( : 
16 
ye tp,mXp,m < bm Tle ett (capacities) 
p=1 
Xp,m = 0 pi eer WG: alee 4 


The objective minimizes total production cost. One system of main constraints en- 
forces product demands and the other mill capacities. 

Table 4.4 shows an optimal solution x*. Old mill 1 should go virtually unused; 
mills 2 and 3 concentrate on pressure and standard tubing, respectively; and new 
mill 4 satisfies the remainder of demand for both. Total weekly cost is $378,899. 


TABLE 4.4 Tubular Products Example Optimum 


Mill1, Mill2, Mill3, Mill 4, 


Product oA X,2 Xp,3 Xh4 
Standard 

1: 3 in. thick 0.0 0.0 0.0 100.0 

2; in. thin 0.0 0.0 630.0 0.0 

3: 1 in. thick 0.0 0.0 404.8 95.2 

4: Lin. thin 0.0 0.0 980.0 0.0 

5: 2 in. thick 0.0 0.0 0.0 720.0 

6 2 in. thin 0.0 0.0 0.0 240.0 

7: 8 in. thick — 0.0 0.0 75.0 

8: 8 in. thin — 0.0 0.0 22.0 
Pressure 

9: $in. thick 0.0 50.0 — 0.0 

10:  $ in. thin 0.0 22.0 — 0.0 

fie inathick 0.0 214.1 — 138.9 

12: 1in. thin 55.0 0.0 — 0.0 

13: 2 in. thick 0.0 125.0 — 0.0 

14: 2 in. thin 0.0 0.0 — 35.0 

15: 8 in. thick — 1OO!Oh, =, — 0.0 

16: 8 in. thin = 10.0 _— 0.0 


EXAMPLE 4.4: CANADIAN FOREST PRODUCTS (CFPL) OPERATIONS 
PLANNING 


Jperations planning models become more complex when there are several stages 
»f production. Activity at each stage consumes output of the preceding stage and 
creates input to the next stage. 

Canadian Forest Products Limited (CFPL) employed such a model to plan 
‘heir production of plywood.’ Figure 4.1 shows the sequence of stages. Production 


4Based on Dilip B. Kotak (1976), “Application of Linear Programming to Plywood Manufacture,” 
terfaces, 7:1, part 2, 56-68. 
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FIGURE 4.1 Plywood Processing Flow in CFPL Example 


begins by purchasing logs and peeling them into strips of thin “green” veneer. Green 
veneer can also be purchased directly. All green veneer is next dried, classified by 
quality, and in some cases improved by patching knots and gluing thin strips together. 
After the veneer has been cut into sheet sizes, several layers are glued and pressed 
to produce plywood. A final production step sands completed plywood and trims it 
to exact size for sale. 

The objective of CFPL’s operations research analysis was to determine how 
to operate production facilities to maximize contributed margin: sales income less 
wood costs. Labor, maintenance, and other plant costs were assumed fixed. The 
principal constraint, other than limits on availability of wood and the market for 
various products, was the limited plant capacity to press plywood. 

To have some numbers to work with, assume that logs are available from two 
vendors in “good” and “fair” Guanes at the rate and price shown below. The table 


also shows the estimated yield in ;¢- and s-inch green veneer of grades A, B, and C 


from peeling a log of the quality indicated. 


Veneer Yield (sq ft) 


Vendor 1 Vendor 2 

Good Fair Good Fair 
Available per month 200 300 100 1000 
Cost per log ($ Canadian) 340 190 490 140 
A ;-inch green veneer (sq ft) 400 200 400 200 
B_ 34-inch green veneer (sq ft) 700 500 700 500 
C 34-inch green veneer (sq ft) 900 1300 900 1300 
A }-inch green veneer (sq ft) 200 100 200 100 
B_ ;-inch green veneer (sq ft) 350 250 350 250 
C_ }-inch green veneer (sq ft) 450 650 450 650 


We can also purchase green veneer. Suppose that availabilities and purchase 
prices are as shown in the following table. 
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a 


7-Inch Green Veneer i-Inch Green Veneer 


A B (G A B Cc 
Available (sq ft/month) 5000 25,000 40,000 10,000 40,000 50,000 
Cost ($ Canadian/sq ft) 1.00 0.30 0.10 2.20 0.60 0.20 
a ea ah eh 


_ Our version of CFPL will make just 6 products—all 4- by 8-foot sheets of plywood 
for the U.S. market. A final table shows the composition of each product in veneer 
sheets, the available market per month, and the time required to glue and press each 
sheet of plywood out of a monthly capacity of 4500 hours. 


;-Inch Plywood Sheets 3-Inch Plywood Sheets 


AB AC BC AB AC BC 
Front veneer tA ZA EB cA nA nB 
3 aC 4C 
Core veneer zC 4C zC 4 iB 4 
3 4C } 
Back veneer is gC ie 6B gC is 
Market per month 1000 4000 8000 1000 5000 8000 
Price ($ Canadian) 45,00 40.00 33.00 75.00 65.00 50.00 


Pressing time (hours) 0.25 0.25 0.25 0.40 0.40 0.40 
er ete EI ee ee A ee Le eS 


CFPL Decision Variables 
As usual, we begin a model for the CFPL case by choosing variables deciding how 
much of what to do. Index dimensions include 

gq & log quality (q = G for good, F for fair) 

v 4 log vendor number (v = 1, 2) 

t A veneer thickness (t = i: 2) 

g 4 veneer grade (g = A,B,C) 


To formulate the problem asa linear program, we will use four classes of (continuous) 
decision variables over these index dimensions: 
Wav & number of logs of quality g bought from vendor v and peeled into green 
veneer of thickness t per month 
X;,g 2 number of square feet of thickness t, grade g green veneer purchased 
directly per month 
Yrg.e’ & number of sheets of thickness ¢ veneer used as grade g’ after drying 
and processing from grade g green veneer per month 
Zi,g,¢’ & number of sheets of thickness f, front veneer grade g, back veneer grade 
g’ plywood pressed and sold per month 


Notice that these variables correspond to only 4 of the 6 processing boxes in 
Figure 4.1. This efficiency is possible because the way we have presented the problem 
offers no advantage for purchasing a log and not peeling it, or pressing a sheet of 
plywood and not sanding or selling it. If inventories had to be modeled, so that for 
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example, purchased logs need not be immediately peeled, we would require many 
more variables. 


Continuous Variables for Integer Quantities 


Readers who are studying LP modeling for the first time may be perplexed about the 
fact that the CFPL decision variables are all treated as continuous. Don’t quantities 
such as the number of logs and the number of sheets of plywood need to be integers? 
Indeed, how can CFPL’s problem even be modeled as a linear program (which must 
have only continuous variables)? 

Modeling physically integer quantities with continuous decision variables in 
this fashion is standard when optimal variable magnitudes are likely to be relatively 
large (principle | 1.11 |). If the LP-optimal number of plywood sheets sold of some 
type turns out to be, say, 953.2, there is little practical difficulty in rounding off to 
953 sheets. After all, the costs, capacities, and other constants in the model are only 
estimates that contain a certain amount of error. 

But we know that there is a big gain in tractability. Continuous optimization 
is almost always more efficient than discrete. To realize that gain without having 
much impact on the usability of optimal results, we choose to neglect integrality 
requirements. 


_4.6 | To gain tractability with little loss of validity, decision variables of rel- 
atively large magnitude are best modeled as continuous, even though they 
correspond to physically integer quantities, 


Notice that this concession to tractability would be much more serious when 
decision variables were limited to, say, 0 and 1. If, for example, 0 means do not build 
a facility and 1 means build it, rounding continuous LP solutions could be much more 
problematic. 


CFPL Objective Function 


CFPL’s maximum contributed margin objective is easily expressed in terms of the 
decision variables above. We compute 


max — (log costs) — (purchased veneer costs) + (sales income) 
That is, 


max — (340we,1,1/16 + 190We1,1/16 + 490WG.2,1/16 + 140Wx2,1/16 (4.6 
+ 340W6,1,1/8 + 190wF1,1/8 + 490weG,2,1/8 + 140wp2,1/8) 
— (1.00%1/16,4 + 0.30%1/16,8 + 0.10x1/16,c + 2.20%X1/8,4 
+0.60%1/3,8 + 0.20x1/8,c) + (4521/4,4,8 + 4021/4,4,C 
+3321/4,8,c + 7521/2,4,8 + 6521/2,4,c + 5021/2,2,c) 


CFPL Constraints 

Some constraints are equally easy. Log availability limits impose 
WG,1,1/16 + WG,1,1/8 = 200, Write + Wr11/3 < 300 
WG2,1/16 + We2,1/3 < 100, wroij6 + We2,1/3 < 1000 
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purchased veneer availabilities imply that 
X116,4 = 5000, X116,8 < 25,000, x116,c < 40,000 


4.8 
X1/8A 3S 10, 000, X13.B S 40,000, X1/8,c S 50,000 ( ) 

and market sizes constrain 
Z1/4,4,B £1000, Z1/4.4,.c < 4000, 2Z1/4,8,c < 8000 (4.9) 


21/2,4,B = 1000, 21/2,4,c < 5000, 21/2,8,c < 8000 
Finally, the important pressing capacity limit yields the additional constraint 
0.25 (21/4,4,.B + 21/4,A,C +21/4,B,C) + 0.40(21/2,4,8 + 21/2,4,C + 21/2,8,c) < 4500 (4.10) 


Balance Constraints 


So far we have done nothing to link log and veneer purchasing at the beginning of 
the process to sales at the end. In fact, we have not used the processing variables 
Yig,g' at all. 

What makes operations planning models with several processing stages special 
is the need to provide such links through balance constraints. 


4.7_| A balance constraint assures that in-flows equal or exceed out-flows fo 
materials = products created by one stage of production and consumed. 


The first family of balance constraints needed in the CFPL model involves green 
veneer. Assume that with trim losses, 35 square feet of green veneer is required for 
each 4- by 8-foot sheet of finished veneer. We then have for each thickness and grade 


(veneer from peeled logs) + (veneer purchased) > 35(sheets of veneer finished) 


Assuming that careful piecing and patching can permit green veneer of one grade 
to be used as the next higher, and veneer of any grade can be substituted for the 
next lower, we obtain the following six balance constraints for various grades and 
thicknesses of green veneer: 


400We,1,1/16 + 200WF1,1/16 + 400WG,2,1/16 + 200WF 21/16 + %1/16,A 
> 35y1/16,4,A + 3591/16,4,B 

700W6,1,1/16 + 500WF1,1/16 + 700WG2,1/16 + 500WF 21/16 + X1/16,8 
> 35y1/16.B,A + 35y1/16,B,B + 3591/16,B,C 

900We.1,1/16 + 1300We 11/16 + 900WE.2,1/16 + 1300WF 2,116 + X1/16,C 
> 35y1/16,0.8 + 35y1/16,0.c 

200We,1,1/8 + 100WA 11/3 + 200WG.2,1/3 + 100Wr21/3 + X1/8,4 
> 35y1/3,4,4 + 35y1/8,4,B 

350w6,1,1/8 + 250Wp1,1/8 + 350WG2,1/8 + 250WF2,1/3 + X1/8,B 
> 35y1/8,8,A + 35y1/8,8,8 + 35y1/8,B,c 

450w6,1,1/8 + 650We1,1/3 + 450WG,2,1/3 + 050WA2,1/8 + X1/8,C 
> 35y18,c.8 + 35y1/8,¢,.c 


(4.11) 


145 


146 


Chapter Four e Linear Programming Models 


Six quite similar constraints enforce balance in sheets of finished veneer passing 
from the drying process to pressing: 


sheets finished for use at this grade = sheets consumed in pressing 


We can make the constraints equalities this time because no veneer would ever be 
finished unless it were going to be pressed. Again detailing for two thicknesses and 
three grades (other than the never-used $-inch, grade A finished veneer) gives 


¥1/16,A,A + Y1/16,B,A 
= 71/4,A,B + 21/4,A,C + 21/2,A,B + Z1/2,A,C 
Y1/16,4,B + Y1/16,B,B + Y1/16,C,B 
= 71/4,A,B + 21/4,B,C + 21/2,4,B + 21/2,B,C 
¥1/16,B,C + Y1/16,C,C (4.12) 
= 21/4,A,C + 21/4,B,C + 21/2,A,C + 21/2,B,C 
¥1/8,A,B + Y1/8,B,B + Y1/8,C,B 
= 71/2,A,B + 21/2,A,C + 21/2,B,C 
Y1/8,B,C 1 Y1/8,C,C 
= 21/4,A,B + 21/4,4,c + 21/4,B,c + 221/2,4,B + 221/2,4,0 + 221/2,B,C 


Use decision variables 
x; 4 number of parts or assemblies j produced 


to formulate balance constraints for assemblies/parts j = 2, 3, 4. 


Modeling: Assembly 1 joins an assembly 2 with two part 3’s and a part 4. Assembly 
2 consists of one part 3 and two part 4’s. Thus for j = 2, we require that the number 
of assembly 2’s at least equal the number required for assembly 1’s: 


X22 x4 


Similarly, for parts j = 3, 4, we require production to at least meet requirements for 
the assemblies: 
x3 > 2x, + 1x2 


X4 > 1x, + 2x2 


CFPL Example Model 


Collecting (4.6)-(4.12) and adding variable-type constraints produces the full CFPL 
linear programming model detailed in Table 4.5. One optimal solution has the 


TABLE 4.5 CFPL Example Linear Program Model 


max 


s.t. 


— 340WG 1116+ 190WF 1 116+ 490WG 2116+ 140WE 2116 
+ 34061 1/3 + 190W p31 1)8+ 490WG 21/3 + 140WF2 1/3) 
- (1.00% 4 /46,4 + 0.30% 116,83 + 0.10%416,c 

+ 2.20x, 18,A + 0.60% 1/8, p + 0.20% 1/3.c) 

+ (4524/4,4,8 + 4021 4,4,c+ 3321/4,.B.C 

+ 7521 2,4,B + 9521 2,4,¢ + 9021 2,2,c) 

Weare t WG11/8 £200, Wri ane + Weiss < 300 
We,2,1/16 + WG,2,1/8 $100, W216 + Wr2,1/8 < 1000 

X 116-4 $5000, X16, 5 $25,000, x1 /16,c $40,000 

X1/g,.1 £10,000, x1 /g, p < 40,000, x1 )5,¢ $50,000 

£1/4,A,B < 1000, Z1/4,A,C S 4000, Z1/4,B,CS 8000 

212,4,B £ 1000, 21 2,4,c $5000, 212,5,¢ $8000 


0.25(21)4,4,B + 21/4,4,C + 21/4,B,c) + 9.40(21 2.4.8 + 2172,4,C + 21/2,8,c) $ 4500 
400W G1 116+ 200W p11 /16 + 400WG 2.1116 + 200W E2116 + ¥1/16,A 


2 35y1/16,4,A + 3591/16,A,B 


1200We 11116 + 500W 4 1/16 + 700WG21/16 + 500WF 21/16 + X1/16,B 


2 35y1/16,8,A + 35¥1/16,8,B + 3591/16,B,C 
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900W G1 1/16 + 1300W Ey 1/16 + 200WG 21/16 + 1300W Eo 1/16 + X1/16,C 


2 35y1/16,c,8 + 35Y1/16,c,c 

200WG 11/3 + L00W A, 1/3 + 200WG2,1/3 + 100WF21/3 + X1/8,4 
2 35y1/3,4,4 + 35Y1/8,4,B 

350WG 1173 + 250Wp) 1/3 + 350WG_2.1/3 + 250Wp21/8 + X1/8,B 
2 35y1/8,2,4 + 35Y1/8,3,8 + 351/8,8,C 

450wG. 1/8 a 650W 11/8 + 450WG21 gt 650Wp 24 Ig X1/8,C 
235y1 8.0.8 + 35y178,.0C 

¥1n6,A,A + Y1/16,B,A = 21/4,4,B + 21/4,A,C + 21/2,4,B + 21/2,4,C 

Y1/16,A,B + Y1/16,B,B + Y1/16,C,B = ©1/4,A,B + 21/4,B,C 
+ 21/2.A,B + 21/2,B,C 

Y16,B,C + Yin6,c,c = 21/4,A,c + 2114,B,C + 21/2,4,C + 2172,B,C 

Y1/8,A,B > ¥1/8,B,B + Y1/8,C,B = 1/2,A,B + 21/2,4,C + 21/2,B,C 

Y178,B,¢ + Y18,¢,c = 21/4,4,B + 21/4,4,c + 21/4,3,c + 221/2,4,8 
+ 221p4c + 22112,3,c 

all variables Wy y 1, X19 Vigo Zr,g,e" 2 9 


following variables nonzero: 


Wot1/16 = 41.3, Wriin6 = 300.0, Wrring = 
Wr1/8 = 844.7, Xt /16.C = 40,000.0, Xt/8,C = 
Yi6,A,A = 3073.2, Yinega = 1329.4, Yinocn = 
Yinecc — 12, 58:4, Yijs,A,B == ALS 5: Yiss.B.c = 
Vice 2989.1, Yiys.cc =14 5127.3; 

Zia = 1000.0, Zigac = 4000.0, Ziyac = 
i abe ai NUL Ore Mp el 


(logs) 
(veneer) 


(sales) 
(log availability) 


(veneer 
availability) 
(market) 


(pressing) 
(green 
veneer 
balance) 


(finished 
veneer 
balance) 


155:3 
50,000.0 
6355.8 
6033.8 


(4.13) 


4355.8 


The firm should enter all markets except the one for $-inch BC plywood. Total 


contributed margin is $484,878 Canadian per month. 


Fractions in such variables as the number of sheets of plywood sold are phys- 
ically impossible. Still, the advantage of globally solving this complex optimization 
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efficiently with linear programming far outweighs the minute inaccuracy associated 
with rounding the LP optimum to obtain a plan. 


TING OPERATIONS PLANNING LPs 


An orange juice company can sell up to 15,000 tons of juice to wholesalers at $1500 
per ton. The juice is either squeezed from oranges purchased at $200 per ton or 
diluted from concentrate obtained at $1600 per ton. Approximately 15,000 tons of 
juice oranges are available and each yields 0.2 ton of juice. The supply of concentrate 
is essentially unlimited, and each ton dilutes into 2 tons of juice. Formulate a linear 
program to choose an operating plan that maximizes the company’s net income (sales 
minus cost). 


Modeling: We define decision variables for each of the 3 operations: 
x; 4 tons of oranges squeezed for juice 
x2 4 tons of concentrated diluted for juice 
x3 4 tons of juice sold 
Then the required model is 
max —200x, — 1600x2 + 1500x3 (net income) 


st. x, < 10, 000 (orange availability) 
x3 < 15, 000 (sales limit) 
0.2x1 + 2x2 = x3 (balance) 


X1,X2,%3 = 0 


Its objective function maximizes the difference of purchase cost and sales rev- 
enue. The first main constraint enforces the limit on orange availability, the second 
limits sales to 15,000, and the third balances production with sales. 


4,4, SHIFT SCHEDULING AND STAFF PLANNING MODELS 


Operations planning models decide what work to undertake so that available re- 
sources are used efficiently. In shift scheduling or staff planning models the work 
is already fixed. We must now plan the resources to accomplish it. In particular. 
we must decide how many of what types of workers and shifts best cover all work 
requirements. Again, LP provides a powerful tool. 


EXAMPLE 4.5: OHIO NATIONAL BANK (ONB) SHIFT SCHEDULING 


The Ohio National Bank (ONB) confronted such a problem in staffing its check 
processing center.” Checks received by the bank already have account numbers an¢ 
other identifying information encoded on them. Machine operators in the check 
processing center key the dollar amount of the check, which is then imprinted with 
the other information for computerized processing. 


>Based on L. J. Krajewski, L. P. Ritzman, and Phil McKenzie (1980), “Shift Scheduling in Banking 
Operations: A Case Application,” Interfaces, 10:2, 1-8. 
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| Checks arrive through the business day in volumes peaking in the early evening. Our 
fictitious version will assume the following arrivals (in thousands): 


Hour Arrivals Hour Arrivals 
11:00 (11 AM.) 10 17:00 (5 PM) 32 
12:00 (noon) 11 18:00 (6 P.M) 50 
13:00 (1 PM.) 15 19:00 (7 P.M) 30 
14:00 (2 P.M.) 20 20:00 (8 P.M) 20 
15:00 (3 P.M.) 25 21:00 (9 P.M) 8 
16:00 (4 PM.) 28 — — 


Uncollected checks cost the bank money in lost interest. Thus it is essential 
that all checks be processed in time for collection on the next business day. ONB 
decided to enforce a requirement that all checks be completed by 22:00 (10 PM.). 
Furthermore, the number unprocessed at any hour should not exceed 20 thousand. 

Two types of employees can perform the check processing task. Full-time 
employees work an 8-hour shift with a 1-hour lunch break in the middle. Part-time 
employees work only 4 hours per day with no lunch. Both types of shifts can begin 
at any hour of the day, and full-time employees can be assigned an hour of overtime. 
Table 4.6 illustrates the possible shifts. 


TABLE 4.6 Possible Shifts in ONB Example? 


Full-Time Shifts Part-Time Shifts 

Start “Sips dan, «-1cdl., ailGGe 4, aij salbvacl7aalik 

fcookerne (athe 220 Rie ae erent he total @asind 4 vote 
2 yg A ah eR ee A a ne, ee 
100. oR oR! oR tok «RE Rate 

PLOO ERS CCRT ESO) TG D fee CHECKS eu! ver ee 
6 00c5 Stes Zl Re IR 
ico R° =| RK =. = RR R R etnies 
Henue its eee a ee ee 
(8-00 iRNGeRIeRN eet aig pet) (SboorN! RNC RNUERN 
19:00 RN RN RN RN RN RN 
20:00 ON RN RN RN RN 
21 Olio =riaONh we Re Odsed es cho fal} ae Ts mete cate Het BN 


*R, regular duty; O, possible overtime; N, night differential. 


In our analysis we assume that full-time employees receive $11 per hour in pay 
and benefits, plus an extra $1 per hour in “night differential” for time after 6 PM. and 
150% pay for daily overtime. Part-time employees are paid $7 per hour, plus $1 per 
sour night differential after 6 PM. Also, to keep overtime under control, we require 
that no more than half the full-time employees on any shift work overtime and that 
‘he total number of scheduled overtime hours not exceed 20 per day. 

Naturally, full-time employees work faster than part-timers. We will assume 
that full-time operators process 1000 checks per hour, and part-timers only 800. 

One final complication is encoding stations. The number of machines available 

mits the number of employees who can work at any one time. Our center will have 
55 machines. 
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ONB Decision Variables and Objective Function 


The main decisions to be made in shift scheduling models are the number of em- 
ployees to work various shifts. In the ONB case we have all the possibilities in 
Table 4.6. For example, the full-time shift starting at 11:00 works 4 hours, then 
takes a lunch break, then works 4 more hours. The final 2 hours come after 6 PM.. 
so a night differential applies. One additional hour may also be worked in over- 
time. 

Using the index 


h A (24-hour clock) shift start time 
we define the corresponding decision variables: 


x, 4 number of full-time employees beginning a shift at hour h 
(hee) 
yn, & number of full-time employees with shift beginning at hour h who work 
overtime (4 = 11, 12) 
Zn & number of part-time employees beginning a shift at hour h 
(oS 5 X18) 
We need only add up the pay for each shift to obtain a minimum (daily) cost 
objective function: 
min 90x11 + 91xq2 + 92x13 + 18yi1 + 18y12 + 28211 + 28212 (4.14 
+ 28213 + 28z14 + 29215 + 30216 + 31z17 + 32218 


For example, the coefficient on x73 reflects 8 regular hours at $11 per hour plus $+ 
for the 4 hours after 6 PM., or 


8($11) + 4($1) = $92 


ONB Constraints 


Table 4.6 also suggests how to model the requirement that no more than 35 operators 
be on duty at any time. We simply constrain the sum of full-time, overtime, and part- 
time employees on duty in each hour. 


Xu + 211 35 (11:00 machines) 
xy + X12 + 211 + Z12 5-35) (12:00 machines) 
; (4.15 
yu + X12 +13 + 217 + 21g < 35 — (20:00 machines) 
Y12 + X13 + Z18 ssh) (21:00 machines) 


There are also overtime limits. Overtime cannot exceed half of any full-time 
shift or total more than 20 hours per day. These limits lead us to the constraints 
yu < 5X11 (11-shift overtime) 


yi <4x1. —_ (12-shift overtime) (4.16 


yutyi2 < 20 (total overtime) 
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Covering Constraints 


The main element in any staff planning model is a collection of covering constraints. 


4.8 | Covering constraints in shift scheduling models assure that the shifts 
chosen provide enough worker output to cover requirements over each time 


With the ONB case we have a slight complication in covering requirements. 
Work arrivals are specified on an hour-by-hour basis, but work completion is limited 
only by all checks being finished at 22:00 (10 pm.). To model covering in such a case, 
we need some new decision variables reflecting the work carried over. Specifically, 
define 


w;, & uncompleted work backlog at (24-hour clock) hour / (in thousands) 


Then our ONB covering constraints take the form 


1x4, + 0.8211 > 10-—wi (11:00 cover) 
day + Lexy. + 0.821, + 0.8212 >1ll+wpe-wye (12:00 cover) 


(4.17) 


1yi + Lxy2 + 1473 + 0.8217 + 0.8213 > 20 + Wr — W21 (20:00 cover) 
Lyy2 + Lxy3 + 0.8218 >8+w (21:00 cover) 


For example, the one for the 20:00 hour requires the total output of workers on duty 
from 20:00 to 21:00 to equal or exceed the 20 thousand checks arriving at that hour 
(see the table at the beginning of Example 4.5), plus checks held over from previous 
hours (w29), less those passed on to later hours (w21). 


ONB Shift Scheduling Example Model 


Combining (4.14)—(4.17) with suitable variable-type constraints and upper bounds of 
20 on all backlog variables w,, produces the full ONB shift scheduling linear program 
shown in Table 4.7. An LP optimum makes the following variables nonzero: 


xt, =8.57, xt, =12.86, yt, =4.29, zt, =13.57, 2%, = 5.36, 
zi, = 7.50 zi, =0.71, wi = 10.00, wi, = 12.43, 
wi, = 6.00, w*,=2.29, wt, =20.00 w%,=17.71, w3, =9.71 


That is, full-time employees carry the load early in the day, with part-time beginning 
at 14:00 (2:00 PM.) Total cost is $2836 per day. ~ 

Once again we have a fractional solution that certainly must be implemented in 
whole numbers of employees. Managers will need to round above LP-optimal values 
to obtain a satisfactory plan. Still, any loss of optimality resulting from rounding will 
fall well within the variability of hourly check arrivals and other data. Unless the 
aumbers of persons working shifts are in the range of, say, 0 to 2, our LP model (4.7) 
sa valid approximation justified by its outstanding tractability. 
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TABLE 4.7 ONB Shift Scheduling Example LP Model 


min 90x11 + 91x49 + 92x;3 + 18y11 + 1849 + 28241 + 28242 (total pay) 
+ 28213 + 28244 + 29245 + 30216 + 31247 + 32zZ4 

St. Xp +214 <35 (11:00 machine) 
Xyy+Xyo4+ 24+ 22 < 35 (12:00 machine) 
Xyy t+ X40 + X43 4 24+ 24 213 <35 (13:00 machine) 
Xyy t+ X42 4+ 2434 Zy.4+ ZF 234 214 35 (14:00 machine) 
XQ + X34 29+ 243+ Z44 Z15 < 35 (15:00 machine) 
Xyy + X34 2134 24+ 215+ 216 <35 (16:00 machine) 
Xyy + Xyo+ Zy4t 25+ 216+ 217 <35 (17:00 machine) 
X44 + Xo + X43 4+ 25+ 216+ 217+ Zig < 35 (18:00 machine) 
Xy, +X. +X13 + 216+ 217+ Zig 35 (19:00 machine) 
Vy t Xo + X43 4+ 217 + Z1g <39 (20:00 machine) 
Yy2 + X13 + Z1g <35 (21:00 machine) 
yy <3xy (11-shift overtime) 
yp < 5x2 (12-shift overtime) 
Yutyi0 < 20 (total overtime) 
1x41 + 0.824, > 10-wy (11:00 cover) 
1x41 + 1x42 + 0.821, + 0.8242 >1l+wp-wy (12:00 cover) 
1xy1 + Uxyo + 1443 + 0.8211 + 0.8247 + 0.8243 > 154+ w73-wWy4 (13:00 cover) 
1x41 + 1xy2 + 1x13 + 0.821, + 0.8242 + 0.8213 + 0.8244 > 20+ wy4- W415 (14:00 cover) 
1x42 + 1x13 + 0.8212 + 0.8243 + 0.8244 + 0.8245 > 254+ Wy5-W16 (15:00 cover) 
1x41 + 1x3 + 0.8213 + 0.8244 + 0.8215 + 0.8246 > 28+ Wig—W17 (16:00 cover) 
1x41 + Lyy2 + 0.8244 + 0.8245 + 0.8246 + 0.8247 > 32+ w17- W318 (17:00 cover) 
Lxq4 + Lyyg + 1x43 + 0.8215 + 0.8246 + 0.8247. + 0.8218 > 50+ Wig-Wy9 (18:00 cover) 
1x41 + 1x49 + 1x13 + 0.8216 + 0.8247 + 0.8248 > 30 + Wy9— Wao (19:00 cover) 
Ly, + Lxyo + 1x43 + 0.8217 + 0.8218 > 20 + Wa)- Wo} (20:00 cover) 
Ly jo + 1x13 + 0.8218 >8+w, (21:00 cover) 


all variables w;, < 20 
all variables wy, Xp. Vp. Zp_ > 0 


Clerical employees of a government agency are allowed to work four 10-hour days 
per week in any of the following patterns: 


Pad Monday—Wednesday—Thursday—Friday 
(Se Monday—Tuesday—Thursday—Friday 
J=3 Monday—Tuesday—Wednesday—Friday 


Formulate a linear program to determine the minimum number of employees needed 
to have at least 10 on duty Mondays, 9 in the office on Fridays, and 7 working on 
Tuesdays through Thursdays. 


Modeling: We employ decision variables 
x; £ number of employees working pattern j 


The required LP model is then 


min x, +%+%3 (total staff) 

S09" Oe og Fa lO (cover Monday) 
X2 + x3 > 7 (cover Tuesday) 
x, +23 Sa) (cover Wednesday) 
xX, +X Sy) (cover Thursday) 
tw t+% => OD (cover Friday) 


1, X2,x3 > 0 


4.5  Time-Phased Models 


The objective minimizes total staff, and the constraints enforce the specified coverage 
on all working days. 


é § TIME-PHASED MODELS 


So far in this chapter we have formulated only static models—those where all plan- 
ning is for a single period of time. Many, perhaps most, linear programs are dynamic 
or time phased because they address circumstances that vary over time. In this sec- 
tion we introduce time-phased modeling. 


EXAMPLE 4.6: INSTITUTIONAL FOOD SERVICES (IFS) CASH FLOW 


LP models of almost any type may require time-phased decision making, but some 
of the most obviously time dependent involve cash flow management.® Every busi- 
ness must keep track of the coming and going of its cash accounts, borrowing where 
necessary and investing when wise. 

We illustrate the modeling issues with a fictional Institutional Food Services 
company (IFS) that supplies food and other products to restaurants, schools, and 
similar institutions. Table 4.8 shows IFS’s projections of some relevant accounts 
over the next 8 weeks (in thousands of dollars). 


s; & projected revenue in week t from cash sales to small customers 

r, & projected accounts receivable revenue received in week ft from large 
customers who buy on credit 

Di & projected accounts payable to IFS’s suppliers in week ¢ 

e, A projected payroll, utility, and other expenses to be paid in week t 


Cash sales and accounts receivable produce immediate income to IFS’s checking 
account. Expenses are immediate deductions. Accounts payable amounts p; are not 
actually due until week ¢ + 3, but they are discounted by 2% if paid early in week t. 


TABLE4.8 IFS Cash Flow Example Data 


Projected Weekly Amount ($ 000’s) for Week: 


Item 1 2 3 4 5 6 , 8 
Cash Sales, s, 600 750 1200 2100 2250 180 330 540 
Accounts receivable, 7, 770 1260 1400 1750 2800 4900 5250 420 
Accounts payable, p, 3200 5600 6000 480 880 1440 1600 2000 
Expenses, é, 350 400 550 940 990 350 350 410 


Values in Table 4.8 vary dramatically over the period as a holiday approaches. 
Besides the option on accounts payable, IFS’s financial officer has two additional 
ways of dealing with the implied cash flow difficulties. First, the company’s bank has 
extended a $4 million line of credit that may be drawn upon at 0.2% interest per 
week. However, the bank requires at least 20% of the current amount borrowed 
to be maintained (without earning interest) in IFS’s checking account. The other 


6Based on A. A. Robichek, D. Teichroew, and J. M. Jones (1965), “Optimal Short Term Financing 
Decision,” Management Science, 12, 1-36. 
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option is investment of excess cash in short-term money markets. IFS can earn 0.1% 
interest per week on amounts invested in this way. 

The financial officer wishes to minimize net total cost in interest and lost dis- 
counts while maintaining at least a $20,000 checking account safety balance. Our 
task is to help him decide how to exercise the available options. 


Time-Phased Decision Variables 


Time is always an index dimension in time-phased models because both input con- 
stants and decisions may be repeated in each time period. For our IFS example, time 
is the only index dimension. Decision variables for the three cash flow management 
options are (in thousands of dollars) 


&t 
h, 


WwW; 


> 


amount borrowed in week t against the line of credit 

amount of line of credit debt paid off in week t¢ 

amount of accounts payable in week t delayed until week t+ 3 ata 
loss of discounts 

x, 4 amount invested in short-term money markets during week t 


x 
as 


For modeling convenience, we also define 


y, & cumulative line of credit debt in week t 
Z: 4 cash on hand during week t 


These variables could be eliminated by substituting suitable sums of the others, but 
many constraints are much easier to express when the extra variables are included. 


Time-Phased Balance Constraints 
Although separate decisions may be made in each period of a time-phased model, 
choices for different periods are rarely independent. Decisions in one period usually 
imply consequences that carry over into the next. 

Such interactions among decisions for different time periods can often be mod- 
eled with balance constraints similar to those of definition 3 


starting . 
level in 


h period t to the next. 


In our IFS example there are two main quantities carried over in this way: cash 
and debt. To develop the required balance constraints, we first enumerate the cash 
increments and decrements each week: 


Cash Increments Cash Decrements 
Funds borrowed in week ¢ Borrowing paid off in week rf 
Investment principal from week t-—1 Investment in week ¢ 
Interest on investment in week ¢-1 Interest on debt in week t- 1 
Cash sales from week t Expenses paid in week t 
Accounts receivable for week t Accounts payable paid with discount for week 


Accounts payable paid without discount for week ¢—3 


4.5 Time-Phased Models 


Using the symbols defined above, these increments and decrements lead to the fol- 
lowing system of balance constraints: 

Zi-1 + Br — Ae + Xp — Xy + 0.001x,_1 — 0.002y,_1 

+ $,—e@ +7, — 0.98(p; — wz) — Wi-3 = Zt Toler CO) (cash balance) 
(All symbols with subscripts outside the range 1,..., 8 are assumed = 0.) 


A similar constraint system tracks cumulative debt. New borrowing increases, 
and paying off decreases: 


W1+8—hi=yt t=1,...,8 (debt balance) 


xq 4 thousands of snow shovels produced in quarter q 
i, 4 thousands of snow shovels held in inventory at the end of quarter q 


to meet customer demands for 11,000, 48,000, 64,000, and 15,000 shovels in quarters 
q=1,...,4, respectively. Write balance constraints in shovels for the four quarters 
assuming inventory at the beginning of the first quarter = 0. 


Modeling: Following principle [4.9], the constraints will have the form 
(beginning inventory) + (production) = (demand) + (ending inventory) 
Now taking initial inventory = 0, we have 


O+x, =11+i (quarter 1) 
iy tx. = 48+h (quarter 2) 
b+x3 = 0644+2 (quarter 3) 
ip t+x4 = 1S4+% (quarter 4) 


IFS Cash Flow Model 


We are now ready to state a full linear programming model for our IFS case: 


8 8 8 
min 0.002 }° y; + 0.02 } > w; — 0.001 purr (net interest) 
t=1 t=1 t=1 
st. Zp-1 + gt — Ae + 1-1 — Xt 
+ 0.001x,_1 one 0.002y;_1 + St = er 


+r; —0.98(p; —wW:) —Wi3=Z% t=1,...,8 (cash balance) 
yit eg —ht =): t=1,...,8 (debt balance) (4.18) 
y, < 4000 t=1,...,8 (credit limit) 
Z, = 0.20y, t=1,...,8 (bank rule) 
We < Dr t=1,...,8 (payables limit) 
Zr = 20 t=1,...,8 (safety balance) 
81, Me, We, Xt, Ve, Ze = O t=1,...,8 (variable type) 


The objective function minimizes interest paid, plus discounts lost, less interest 
earned. Besides the two systems of balance requirements formulated above, con- 
straints enforce the credit limit and the bank rule requiring that 20% of borrowed 
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funds be kept as cash, keep delayed accounts payable within the value from Table 
4.8, and ensure a continuing safety balance of $20,000. All symbols with subscripts 
outside the range 1,..., 8 are assumed = 0. 

Table 4.9 presents an optimal solution. The corresponding optimal net interest 
and discounts total $158,492 for the 8 weeks. 


TABLE 4.9 IFS Cash Flow Optimal Solution 


Optimal Weekly Amount ($ 000’s) for Week: 


Decision Variable 1 2 3 4 5 6 7 8 
Borrowing, g; 100.0 505.7 3394.3 0.0 442.6 0.0 0.0 0.0 
Debt payment, h, 0.0 0.0 0.0 442.5 0.0 2715.3 1284.7 0.0 
Payables delayed, w, 2077.6 3544.5 1138.5 0.0 0.0 0.0 0.0 0.0 
Short-term investments, x, 0.0 0.0 0.0 0.0 0.0 0.0 2611.7 1204.3 
Cumulative debt, y, 100.0 605.7. 4000.0 3557.4 4000.0 1284.7 0.0 0.0 
Cumulative cash, z, 20.0 121.1 800.0 711.5 800.0 256.9 20.0 20.0 
Time Horizons 
A time horizon establishes the range of time periods in a time-phased model. For 
example, our IFS cash flow example uses a fixed time horizon of 1, ..., 8 because we 


model only 8 weeks. 

Of course, IFS would have been operating before the current 8-week period 
and will continue operations after it. Thus our use of a fixed time horizon raises some 
special concerns about periods near the boundary. 

In particular, model (4.18) assumes that all quantities outside the time horizon 
equal zero. Thus IFS begins week 1 with zero cash balance zo, zero debt yo, and zero 
short-term investments x. Optimal results in Table 4.9 could change dramatically 
if some of these boundary values proved to be nonsensical. To obtain,a more valid 
model, it might be necessary to estimate typical values and include them as constants 
in balance equations for week 1. 

At the other end of the time horizon we have similar issues. Although the 
optimum in Table 4.9 chose not to delay accounts payable in the last few weeks, it 
might have been severely tempted. Payables delayed in the last 3 weeks never have 
to be paid because the due date (t + 3) falls beyond the time horizon. 

Such issues do require particular attention if models with time horizons are to 
produce valid results. 


One way to avoid having to think about boundaries of a fixed time horizon is 
to employ an infinite time horizon model. Infinite horizon schemes “wrap around” 
output states of a last time period as input conditions for the first. The result is that 
time goes on infinitely even though only a few periods are modeled explicitly. 


4.6 Models with Linearizable Nonlinear Objectives 


Infinite horizon modeling of our IFS example would treat ¢ = 1 as the period 


immediately after week t = 8. Then, for example, the debt balance constraint for 
t = 1 would read 


yetgi-m=y1 


Return to the snow shovel problem of Sample Exercise 4.7, and write balance con- 
straints for the four quarters under each of the following assumptions about inventory 
at time horizon boundaries. 


(a) The time horizon is a fixed four quarters, with beginning inventory in the first 
quarter of 9000 shovels. 

(b) The time horizon is infinite, with quarter 1 following quarter 4. 

Modeling: 


(a) With initial inventory = 9, the required balance constraints are 


94x, = 11+ (quarter 1) 

ip tx. = 48+ (quarter 2) 

in +x3 = 64+% (quarter 3) 

+x4 = 15+ (quarter 4) 
(b) With inventory wrapped around from the last to the first quarter, the balance 
constraints are 

ig +x, = 1144 (quarter 1) 

tw = 484+h (quarter 2) 

bn +x3 = 64+% (quarter 3) 

R+x4 = 15+ (quarter 4) 


4.6 MODELS WITH LINEARIZABLE NONLINEAR OBJECTIVES 


Because LP models possess all the tractable features explored in Chapter 3, a lin- 
ear programming model of a problem is almost always preferable to a nonlinear 


one of equal validity (principle | 2.31 | of Section 2.4). Nonlinearity is often un- 
avoidable, but some frequently occurring nonlinear objective functions are excep- 
tions. We introduce in this section those minimax, maximin, and min deviation 
objective functions which though nonlinear at first glance can be modeled with a 
linear objective function and linear constraints. Interested readers may also wish 


to refer to the related discussion of separable nonlinear programming in Sec- 
tion 14.8. 
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EXAMPLE 4.7: HIGHWAY PATROL 


We begin with a real allocation problem addressed by the Highway Patrol of a south- 
ern state.’ The Patrol wished to divide the effort of its on-duty officers among 
highway segments in each territory to maximize speeding reduction. 

The first two lines of Table 4.10 illustrate the types of data available. Highway 
segments in our fictitious version are indexed by 


j & highway segment number (j = 1,..., 8) 
with 25 officers per week to allocate. Analysts were able to estimate for each seg- 
ment: 


u; & upper bound on the number of officers assigned to segment 
j per week 

r; & reduction potential for suppressing speeding on segment 
j per officer assigned 


A high reduction potential indicates a segment where a patrol would be especially 
effective. In the real application, reduction potentials were obtained by directly 
measuring segment traffic speeds with and without an officer on patrol. 


TABLE 4.10 Highway Patrol Example Data and Solutions 


Values by Highway Segment, j 


—————ee ee ee re ee 
1 2 3 4 5 6 qi 8 
a 
Upper bound, u; 4 8 5 ff 6 5 6 4 
Reduction potential, r; 11 3 4 14 2 19 10 13 


Maxisum optimum, x} 4.00 0.00 0.00 7.00 0.00 5.00 5.00 4.00 


Maximin optimum, x; 1.09 4.00 3.00 0.86 6.00 4.85 1.20 = 4.00 


Maxisum Highway Patrol Example Model 


It is obvious that decision variables in our Highway Patrol allocation example should 
be 


x; & number of officers per week assigned to patrol segment j 


Then a straightforward linear programming model is 


8 
max ae FX; (total reduction) 
j=l 
8 
s.t. oy Gp 25, (officers available) (4.19) 
iF 


SS Up Sh Seah (upper bounds) 

xj = 0 Heeler rene (nonnegativity) 
The objective function is a maxisum because it seeks to maximize the sum of 
reductions in different segments (the analog in a minimize model is called minisum). 


Based on Don T. Phillips and Gary L. Hogg (1979), “The Algorithm That Converged Too Fast.” 
Interfaces, 9:5, 90-93. 
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Main constraints restrict solutions to the 25 available officers and enforce upper 
bounds. The third line of Table 4.10 shows an optimal solution to maxisum model 
(4.19) that yields a total speed reduction of 339. 


Minimax and Maximin Objective Functions 


Notice that all but one of the maxisum optimal values in Table 4.10 are either zero 
or upper bound u;. A little contemplation will reveal that this must always happen 
in a maxisum model with constraints as simple as those of (4.19). 

Sometimes we would prefer a minimax or a maximin objective to spread the 
allocation more evenly. 


4, 12 7 Minimax (minimize the fe or ae (mpamive the minimum) 
: b trather than 


Instead of optimizing total output, we focus on the model element with the least 
satisfactory result. 


Nonlinear Maximin Highway Patrol Example Model 


Adopting the maximin approach in our Highway Patrol example yields the model 


ma G47.) Xe minX) 7 = pas,.8} (maximin reduction) 
8 
Sey, ayes (officers available) 
‘= (4.20) 
CR a el eee (upper bounds) 
x; > 0 al AO (nonnegativity) 


The objective now maximizes the least reduction among all highway segments. 


Notice that (4.20) is a nonlinear program (definition [2.14]). Constraints re- 
main linear, but the objective function is no longer a weighted sum of the decision 
variables. Still, this NLP may provide more valid results than maxisum LP (4.19) 
because speed reduction is addressed on every highway segment. The final line of 
Table 4.10 shows that the optimal allocation in this maximin model is much more 
uniform across segments. The specified optimum yields a reduction of at least 12 on 
every segment. 


Linearizing Minimax and Maximin Objective Functions 


With a model as simple as (4.20), nonlinearity may not produce much loss of tractabil- 
ity. Happily, we need not sacrifice tractability even in much more complicated cases. 
By a suitable modification of nonlinear form (4.20), we can formulate an exactly 
equivalent linear program. 

The idea is simply to introduce a new continuous variable 


f 4 objective function value 


and maximize f subject to a system of linear constraints keeping f no more than any 
term of the minimum. 
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and then minimizing f subject to f ach n max element i ina minimax, or max- 
imizing f subject to f < each min element ina a maximin. 


Linearized Maximin Highway Patrol Example Model 


Applying principle to the maximin version of our Highway Patrol model yields 
linear program 


max f (maximin reduction) 
Si ies; i llgoerpte: f <each term 
8 
ps DGS PED) (officers available) (4.21) 
- te Uj ial eee (upper bounds) 
xj 20 jh, eee (nonnegativity) 


Unrestricted variable f is now the only term of the objective function, which makes 
the objective trivially linear. New linear constraints keep f less than or equal to all 
terms 7;x;. 
Transformation [443 | works because any optimal solution in (4.21) must have 

f* = min{rjx; 37 =1,...,8} 


The new system of constraints keeps 


f* < min{rjx7 sj =1,..., 8} 
and an f strictly less than the minimum r;x; can be increased to improve the objec- 
tive. 
SAMPLE EXERCISE 4.9: 


In terms of decision variables x1, x2, and x3, the production times required on a 
company’s two assembly lines are 


3x, +2x2.+1x3 and 1x; + 5x2 


Assuming that all other constraints are linear, formulate an objective function and 


extra constraints needed to minimize the maximum time on the two lines as a linear 


program. 
Modeling: Following principle | 4.13 | for a minimax case, we introduce new unre- 
stricted variable f and employ objective function 

minf 
To make this minimize the maximum, we also add constraints 


f = 3x1 + 2x2 + 1x3 
i = 1x, ais 5x2 
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EXAMPLE 4.8: VIRGINIA PRESTRESS (VP) LOCATION 


To see another common nonlinear objective that can be linearized, we consider the 
location problem confronted by a firm we will call Virginia Prestress (VP).° VP was 
planning for production of a new product: concrete utility poles. That production 
required a new concrete casting area to make poles and a storage area for finished 
products. 

Figure 4.2 presents the implied facilities location problem for our fictitious case. 
The two new facilities will interact with each other and with three existing operations: 
the concrete batching facility, where premixed concrete is prepared, the steel area, 
where reinforcing steel is manufactured, and the shipping gate, where finished poles 
are processed out of the plant. A coordinate system quantifies the locations of all 
three existing facilities, and the adjoining table displays material handling costs of 
expected traffic between facilities. For example, each foot of distance between the 
pole storage area and the shipping gate adds $0.40 in crane activity. We must choose 
locations for the new facilities to minimize total material handling cost. 


concrete 
batching 
1200 rs 
Material Handling Pole Pole 
Costs per Foot Casting Storage 
J = 1 J = 2 
: 1: Pole casting — — 
pole casting 2: Pole storage 4.00 _ 
pole storage 3: Concrete batch 1.10 — 
_ 4: Steel manufacturing 0.70 0.65 
5: Shipping — 0.40 
shipping 
3 
300 600 


FIGURE 4.2 Virginia Prestress (VP) Location Example 


Nonlinear VP Location Model 
It should be clear that the main decisions to be made in this VP case are 


oe 4 x-coordinate of new facility j’s location 
y; & y-coordinate of new facility j’s location 


We want to choose x1, y1, x2, and y2 to minimize the sum of implied distances to other 
facilities times unit material handling costs. Using values and coordinates in Figure 
4.2, a model is 


min 4.00d(x1, y1, x2. y2) + 1.10d(x1, y1, 300, 1200) 
+0.70d(x1, yi, 0, 600) + 0.65d(x2, y2, 0, 600) (handling cost) (4.22) 
+ 0.40d(x2, y2, 600, 0) 


8Based on Robert F. Love and Lowell Yerex (1976), “An Application of a Facilities Location Model 
in the Prestressed Concrete Industry,” /nterfaces, 6:4, 45-49. 
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where 
A(X}, Yj, Xk, Vk) & distance from (xj, yj) to (Xe, Ye) 


No constraints are required. 

If we measure distance in the straight-line or Euclidean way depicted in Figure 
4,3(a), model (4.22) is unavoidably nonlinear.’ However, it is often more appropri- 
ate in facilities design to calculate distance in the rectilinear manner of Figure 4.3(b). 
Material movements tend to follow aisles and other paths aligned with either the x 
or the y axis. Thus travel distance is best modeled as 


(Xj, Yj, Xk, Ve) & xj — Xe + [7 — Yel (4.23) 


Euclidean 
distance 


Rectilinear 
distance 


(b) 


FIGURE 4.3 Euclidean versus Rectilinear Distance 


Min Deviation Objective Functions 


With rectilinear distance measure (4.23), mathematical program (4.22) takes a min 
deviation form. 


imize eee 
quantities. 


Here we seek to minimize the cost-weighted sum of location coordinate differ- 
ences. 


Linearizing Min Deviation Objective Functions 


Any min deviation objective involving positive-weighted absolute differences of lin- 
ear functions can be modeled linearly. We need only introduce new deviation vari- 
ables expressing the required differences. 


°See, for example, the DClub model of Section 3.1. 
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415 | Positive-weighted ter 
tion involving differences of lini 
linearly by (1) introducing new n 
adding new constraints — 


_and (3) 
Linearized VP Location Model 
With distance rectilinear, our VP location model (4.22) becomes 

min 4.00|x1 — x2| + 4.00|y1 — y2| + 1.10|x; — 300| 
+1.10|y; — 1200] + 0.70|x; — 0| + 0.70|y1 — 600| 


+0.65|x2 — 0| + 0.65|y2 — 600] + 0.40|x2 — 600] 
+ 0.40|y2 — 0| 


To apply principle , we introduce a pair of deviation variables for each objective 
function termi = 1,..., 10: 
s* 4 positive difference in absolute value term i 


s; A negative difference in absolute value term i 


Then we may solve VP’s location problem with linear program 
min 4.00(sf +s; ) + 4.00(s} +57) + 1.10(s} +55) 
+ 1.10(sf +s,) + 0.70(s¢ + sz) + 0.70(sf + 56) 
+ 0.65(s; +57) + 0.65(s¢ + 5g) + 0.40(s$ + 55) 
+ 0.40(s7) +57) 


on ee 7 a == (term 1) 
yi-y2 = Sp sy (term 2) 
x1 —-300 = sj —sz (term 3) 
y1 — 1200 = sj —sy (term 4) (4.24) 
x; —0 = So Hoe (term 5) 
yi — 600 = sp —s, (term 6) 
2-0 = s7 57 (term 7) 
y2 -600 = sy —sy (term 8) 
x2 -600 = sy —s (term 9) 
y2 —0 = "Sy = 8p (term 10) 


Sue oO P< biter rie 0) 


Notice that new (linear) constraints express each absolute difference term of the 
min deviation objective function as the difference of corresponding deviation vari- 
ables subject to nonnegativity constraints. Then the objective function minimizes 
‘he weighted sum (not the difference) of those variable pairs. 
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Here an optimal solution locates the new facilities next to each other at 


x7 = x3 = 300 
yi = ¥, 00 
Corresponding optimal values for deviation variables are 
gf 0h 0) = 0, 30; al* = 300 
57° 0 Sig 0 is 10) = OOF see 0 
5. = 0, = 3007 yy, = ON) sg 0), 535 = 000 


Why does transformation work? Because it can never be optimal for any 
term to have both s* > 0 ands, > 0. For example, at optimality the fourth term of 
our VP model (4.24) has the corresponding constraint 

yi — 1200 = 600 — 1200 = —600 = sj — 5, 


Many choices of s; ands; will satisfy the constraint, but the objective function prefers 
the one with the smallest sum, that is, 


s; = 0, 5," = 600 


With at most one member of each deviation variable pair being positive in an optimal 
solution, their difference will always exactly equal the required absolute difference. 


vehicle can be expressed as 
4x, —xX.+7x3 and 9x, — 10x. + x3 


respectively. Assuming that all other constraints are linear, formulate an objective 
function and extra constraints needed in a linear program to find the design with 
speed as close as possible to 100, and weight as close as possible to 150. 


Modeling: Following principle , we define deviation variables s} and s; for 
speed, together with s} and sy for weight. Then we minimize the deviation of speed 
and weight from their desired values with objective function 


min sj +s; +57 +57 (total deviation) 
and special constraints 
Ax — xp OO = oh (speed) 
9x1 = lOxaterar 150) iste Ss (weight) 
Spey weney oe 


EXERCISES 


4-1 Bisco’s new sugar-free, fat-free chocolate 
squares are so popular that the company cannot keep 
up with demand. Regional demands shown in the 
following table total 2000 cases per week, but Bisco 
can produce only 60% of that number. 


NE SE MW WwW 
Demand 620 490 510 380 
Profit 1.60 140 1.90 1.20 


The table also shows the different profit levels per 
case experienced in the regions due to competition 
and consumer tastes. Bisco wants to find a maximum 
profit plan that fulfills between 50 and 70% of each 
region’s demand. 
a) [Q] Formulate an allocation LP to choose an op- 
timal distribution plan. 
b) [6] BEnter and solve your model with the class 
optimization software. 


4-2 A small engineering consulting firm has 3 senior 
designers available to work on the firm’s 4 current 
projects over the next 2 weeks. Each designer has 
80 hours to split among the projects, and the follow- 
ing table shows the manager’s scoring (0 = nil to 
100 = perfect) of the capability of each designer to 
contribute to each project, along with his estimate of 
the hours that each project will require. 


Project 
Designer 1 2 3 4 
1 90 80 10 50 
2 60 70 50 65 
3 70 40 80 85 
Required 70 50 85 35 


The manager wants to assign designers to maximize 
total capability. 


a) Formulate an allocation LP to choose an opti- 
mal work assignment. 

d) Enter and solve your model with the class 
optimization software. 


4-3 Cattle feed can be mixed from oats, corn, al- 
‘alfa, and peanut hulls. The following table shows 
‘he current cost per ton (in dollars) of each of these 
agredients, together with the percentage of recom- 
mended daily allowances for protein, fat, and fiber 
“at a serving of it fulfills. 
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Oats Corn Alfalfa Hulls 
% Protein 60 80 op) 40 
% Fat 50 70 40 100 
% Fiber 90 30 60 80 
Cost 200 150 100 de 


We want to find a minimum cost way to pro- 
duce feed that statisfies at least 60% of the daily al- 
lowance for protein and fiber while not exceeding 
60% of the fat allowance. 


(a) [| Formulate a blending LP to choose an opti- 
mal feed mix. 

(b) EQ] Which of the constraints of your model are 
composition constraints? Explain. 

(c) &] Enter and solve your model with the class 
optimization software. 


4-4 Several forms of gasoline are produced during 
the petroleum refining process, and a last step com- 
bines them to obtain market products with specified 
quality measures. Suppose 4 different gasolines are 
available, with values for the 2 indexes of quality be- 
ing 99 and 210, 70 and 335, 78 and 280, and 91 and 265, 
respectively. Using corresponding costs per barrel of 
$48, $43, $58, and $46, we would like to choose a min- 
imum cost blend with a first quality index between 
85 and 90 and a second index between 270 and 280. 


(a) Formulate a blending LP to choose an optimal 
gasoline blend. 

(b) Which of the constraints of your model are com- 
position constraints? Explain. 

(c) [| Enter and solve your model with the class 
optimization software. 


4-5 Ronnie Runner distilleries blends i = 1,...,m 
scotch whiskeys to create its j = 1,...,m products 
with properties k = 1,...,p. Unblended whiskey i 
measures a; on scale k. Express each of the follow- 
ing as linear constraint(s) in these parameters and 
the nonnegative decision variables x,; 4 barrels of 
whiskey i used in product 7. Assume that the prop- 
erties combine in proportion to volume and that the 
total production of any blend is free to vary with the 
optimization. 


(a) Property k = 11 should fall between 45 and 
48 in all products. 

(b) Product j = 14 must have all properties k = 
5,...,9 between 90 and 95. 


166 


(c) [Q] Product j = 26 must measure at least 116 on 
property k = 15. 

(d) No product should measure more than 87 on 
property k = 8. 

(ec) [&] Products 6 through 11 should combine input 
whiskey 1 with others in at most the ratio 3:7. 

(f) Input whiskeys 4 and 7 should be in ratio 2:3 for 
all blends. 

(g) At least one-third of all output must come 
from inputs i = 3,..., 6. 

(h) No more than 5% of all output can come from 
input i = 13. 


4-6 Problems are often modeled as linear programs 
even though some decision variables represent quan- 
tities such as the number of units processed or the 
number of times an alternative is used that must be 
integer in a physical implementation. Briefly justify 
this practice. 


4-7 A metalworking shop needs to cut at least 37 
large disks and 211 small ones from sheet metal rect- 
angles of a standard size. Three cutting patterns are 
available. One yields 2 large disks with 34% waste, 
the second gives 5 small disks with 22% waste, and 
the last produces 1 large and 3 small disks with 27% 
waste. The shop seeks a minimum waste way to fulfill 
its requirements. 


(a) [&] Formulate an operations management LP to 
choose an optimal cutting plan. 

(b) [§] Enter and solve your model with the class 
optimization software. 


4-8 Classic Candles handmakes three models of el- 
egant Christmas candles. Santa models require 0.10 
day of molding, 0.35 day of decorating, and 0.08 day 
of packaging and produce $16 of profit per unit sold. 
Corresponding values for Christmas trees are 0.10, 
0.15, 0.03 and $9, while those of gingerbread houses 
are 0.25, 0.40, 0.05, and $27. Classic wants to maxi- 
mize profit on what it makes over the next 20 working 
days with its 1 molder, 3 decorators, and 1 packager, 
assuming that everything made can be sold. 


(a) Formulate an operations management LP to 
choose an optimal production plan. 

(b) ©)Enter and solve your model with the class 
optimization software. 


4-9 Wobbly Office Equipment (WOE) makes two 
models of tables for libraries and other university 
facilities. Both models use the same tabletops, but 
model A has 4 short (18-inch) legs and model B has 
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4 longer ones (30-inch). It takes 0.10 labor hour to 
cut and shape a short leg from stock, 0.15 labor hour 
to do the same for a long leg, and 0.50 labor hour to 
produce a tabletop. An additional 0.30 labor hour 
is needed to attach the set of legs for either model 
after all parts are available. Estimated profit is $30 
for each model A sold and $45 for each model B. 
Plenty of top material is on hand, but WOE wants to 
decide how to use the available 500 feet of leg stock 
and 80 labor hours to maximize profit, assuming that 
everything made can be sold. 


(a) [6] Formulate an operations management LP to 
choose an optimal plan using the decision vari- 
ables x; 4 number of model A’s assembled and 
sold, x, 4 number of model B’s assembled and 
sold, x; 4 number of short legs manufactured, 
x4 4£ number of long legs manufactured, and 
xs 4 number of tabletops manufactured. 

(b) [Q] Which of the constraints of your model are 
balance constraints? Explain. 

(c) &| SEnter and solve your model with the class 
optimization software. 


4-10 Perfect Stack builds standard and extralong 
wooden palettes for a variety of manufacturers. 
Each model consists of 3 heavy separators of length 
equal to the palette. The standard model has 5 cross 
pieces above and 5 below the separators and requires 
0.25 hour to assemble. The extralong version has 9 
similar cross pieces on top and bottom and consumes 
0.30 hour to assemble. The supply of wood is essen- 
tially unlimited, but it requires 0.005 hour to fabri- 
cate a standard separator, 0.007 hour to fabricate an 
extralong separator, and 0.002 hour to fabricate a 
cross piece. Assuming that it can sell as many stan- 
dard models as can be made at $5 profit each and 
as many extralongs at $7 profit, Perfect wants to de- 
cide what to produce with the available 200 hours of 
assembly time and 40 hours of fabrication. 


(a) Formulate an operations management LP te 
choose an optimal plan using the decision vari- 
ables x; 4 number of standard palettes assem- 
bled and sold, x. 4 number of extralongs assem- 
bled and sold, x3; 4 number of standard separa- 
tors manufactured, x4 A number of extralong 
separators manufactured, and x5 4 number of 
cross pieces manufactured. 

(b) Which of the constraints of your model are bal- 
ance constraints? Explain. 

(c) &] Enter and solve your model with the class 
optimization software. 


4-11 Goings Engine produces diesel engines and 
assemblics i = 1,...,m at its plants p = 1,...,n. 
There is some end demand d;, for the various en- 
gines and assemblies, with the rest used in Goings 
production. The number of subassemblies i required 
to produce each assembly k is a; x. 


(a) [Q] Write a system of linear constraints specify- 
ing that the number of each engine and assembly 
must balance (with zero inventories) across the 
company using the parameters above and the 
nonnegative decision variable x;, 4 number of 
assemblies i produced at plant p. 

(b) Write a system of linear constraints specifying 
that the number of each engine and assembly 
must balance (with zero inventories) at each 
plant using the foregoing parameters and the 
nonnegative decision variable x;,,, 4 number 
of assemblies i produced at plant p for use at 
plant q. 


4-12 The River City Police Department uses work 
shifts in which officers work 5 of the 7 days of the 
week with 2 successive days off. For example, a shift 
might work Sunday through Thursday and then have 
Friday and Saturday off. A total of 6 officers must 
de on duty Monday, Tuesday, Wednesday, and Thurs- 
day; 10 are required on Friday and Saturday; and 8 
are needed on Sunday. River City wants to meet 
these staffing needs with the minimum total number 
of officers. 


a) [] Formulate a shift scheduling LP to select an 
optimal staffing plan. 

5) [&] Which of the constraints of your model are 
covering constraints? Explain. 

c) [|] GEnter and solve your model with the class 
optimization software. 


+13 Mama’s Kitchen serves from 5:30 A.M. each 
morning until 1:30 pM. Tables are set and cleared by 
cusers working 4-hour shifts beginning on the hour 
rom 5 A.M. through 10 A.M. Most are college students 
sho hate to get up in the morning, so Mama’s pays 
7 per hour for the 5, 6, and 7 AM. shifts, and $6 per 
sour for all others. The manager seeks a minimum 
sost staffing plan that will have 2 busers on duty for 

te hour beginning at 5 A.M., plus 3,5,5,3,2,4, 6 and 
> on duty for the hours to follow. 


2) Formulate a shift scheduling LP to select an op- 
timal staffing plan. 

>) Which of the constraints of your model are cov- 
ering constraints? Explain. 
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(c) | &Enter and solve your model with the class 
optimization software. 


4-14 The MacKensie’s daughter will begin college 4 
years from today. Her parents want to invest $10,000 
at the beginning of each of the 4 years to accumulate 
a fund that can help pay the cost. Each year they 
expect to have available both certificates of deposit 
returning 5% after 1 year and ones returning 12% 
after 2 years. This year, they also have an opportu- 
nity to make a special investment that would return 
21% after 4 years. The MacKensies want to choose 
investments to maximize their college fund assuming 
that all funds are reinvested at maturity. 


(a) [] Formulate a time-phased LP to choose an 
optimal investment plan. 

(b) [| Which of the constraints in your model are 
balance constraints? Explain. 

(c) EQ] Whatis the time horizon of your model? Ex- 
plain. 

(d) Enter and solve your model with the class 
optimization software. 


4-15 Global Minimum manufactures bikini swim- 
ming suits. Their business is highly scasonal, with ex- 
pected demands being 2800, 500, 100, and 850 dozen 
suits over the four quarters of next year. The com- 
pany can produce 1200 dozen suits per quarter, but 
inventories must be built up to meet larger demands 
ata holding cost of $15 per dozen per quarter. Global 
wants to meet demand while minimizing this inven- 
tory cost. 


(a) Formulate a time-phased LP to choose an opti- 
mal production plan assuming an infinite time 
horizon. 

(b) Which of the constraints in your model are bal- 
ance constraints? Explain. 

(c) Explain why your model has an infinite horizon 
even though it details only four quarters. 

(d) [| Enter and solve your model with the class 
optimization software. 


4-16 A company manufactures parts i = 1,...,m 
in weeks t = 1,...,, where each unit of part i re- 
quires a;, units of production resource k = 1,...,¢ 


and has value v;. Production resource capacities b, 
cannot be exceeded in any period and part demands 
di, must be met. Express each of the following as 
linear constraint(s) in these parameters and the non- 
negative decision variables x;, 4 number of units of 
i produced in week ¢t and z,, A inventory of product 
i held at the end of period ¢. Initial inventories all 
=i) 
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(a) [| No production capacity can ever be ex- 
ceeded. 

(b) The total value of held inventories should never 
exceed 200. 

(c) [Q] Quantities of each parti available after week 
1 should balance with demand and accumulated 
inventory. 

(d) Quantities of cach part i available after weeks 
2,...,2 —1 should balance with demand and 
accumulated inventory. 


4-17 The following table shows observed electrical 
power consumption at several different levels of a 
factory’s operation. 


Level Dri SS ES Fe 7. 
Power 2o=3%°3 5 


Engineers want to fit the estimating relationship 
power = fp + f,level 


to these data in a way that minimizes the sum of 
the absolute deviations between predicted and ob- 
served power requirements. Both parameters fy and 
B, should be nonnegative. 


(a) [8] Formulate a linearized nonlinear LP to 
choose optimal parameter values. (Hint: Bo and 
f, are among the decision variables.) 

(b) [| GEnter and solve your model with the class 
optimization software. 

(c) [J Determine from your solution in part (b) the 
absolute deviation at each observed point. 


4-18 The following figure 


NY 


(10, 15) 
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shows the ceiling locations of 3 sensors in a new fac- 
tory relative to a coordinate system (in feet) with 
origin at the lower left. A control box will be lo- 
cated along the long (lower in the figure) wall with 
fiber-optic cables running rectilinearly to each sen- 
sor. Designers want to place the box to minimize the 
cable required. 


(a) Formulate a linearized nonlinear LP to choose 
an optimal placement. 

(b) [6] GEnter and solve your model with the class 
optimization software. 

(c) How much cable would be required to imple- 
ment your solution of part (b)? 


4-19 [6] Repeat Exercise 4-17, this time choosing a 
fit that minimizes the maximum deviation between 
observed and predicted power requirements. 


4-20 Repeat Exercise 4-18, this time minimizing the 
length of the longest cable. 


4-21 The American Edwards Laboratories (AEL)’ 
manufactures artificial human heart valves from pig 
hearts. One of the things making planning complex 
is that the size of pig hearts is highly variable, de- 
pending on breed, age when slaughtered, feed mix. 
and so on. The following (fictitious) table shows the 
fraction of hearts from suppliers j = 1,...,5 yield- 
ing each of the valve sizes i = 1,...,7, along with 
the maximum quantity available from each supplies 
per week and the unit cost of hearts obtained. 


Supplier j 
Size 1 2 3 4 5 
il 0.4 0.1 — — — 
y 0.4 0.2 — — — 
3 0.2 0.3 0.4 0.2 — 
4 — 0.2 0.3 0.2 — 
5 — 0.2 0.3 0.2 0.2 
6 —_— —_— — 0.2 0.3 
7 — — — 0.2 0.5 
Availability 500 330 150 650 300 


Cost Did 32) 3.0 aad 3.9 


AEL wants to decide how to purchase hearts to mes? 
weekly requirements of 20 size 1, 30 size 2, 120 size 
3, 200 size 4, 150 size 5, 60 size 6, and 45 size 7 valves 
at minimum total cost. 


Based on S. S. Hilal and W. Erikson (1981), “Matching Supplies to Save Lives: Linear Programming 


the Production of Heart Valves,” Interfaces, 11:6, 48-55. 


(a) KS] Formulate an LP model of this heart pur- 
chase planning problem using the decision vari- 
able (j =1,...,5) 

x; & number of hearts purchased weekly from 
supplier j 


(6) &] BEnter and solve your model with the class 
optimization software. 


4-22 Midville Manufacturing assembles heavy duty 
materials handling carts to meet demand of 500 units 
in the first quarter of each year, 1200 in the second, 
1000 in the third, and 300 in the fourth. Elemen- 
tary components, which consist of wheels, steering 
yokes, and carrying platforms, are first assembled 
separately. Then each steering yoke is equipped with 
4 wheels to form the front-end subassembly. Finally, 
front-end subassemblies are combined with a carry- 
ing platform and 8 additional wheels at the rear to 
complete the cart. Using j = 1 for steering yokes, 

= 2 for wheels, j = 3 for platforms, j = 4 for 
‘front-end assemblies, and j = 5 for finished carts, 
the following table shows the estimated value of each 
element (in dollars) and the factory hours required 
to assemble it. 


Element j 
be deed! tin paghibc Sores 
120 40 75 400 700 
0.06 0.07 0.04 0.12 0.32 


Value 


Time 


-omponents, subassemblies, and finished carts pro- 
Suced in any quarter may be used or shipped in the 
‘ame quarter or held over as inventory (including 
rom the fourth quarter to the first) at 5% per quar- 
et interest on the held value. Midville seeks a plan 
»at minimizes these holding costs while conforming 

the factory production capacity, 1150 hours per 
guarter. 


2) Formulate an LP model to choose a production 
plan using the decision variables (j = 1,...,5; 
ilisassauces4) 

Xjq & number of units of element j produced 
in quarter q 


hyq & number of units of element j held in 
inventory from quarter q to the next 
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Your model should include a system of main 
constraints for production capacity and an 
additional system for each element j to enforce 
material balance. 

(b) [6] &[Enter and solve your model with the class 
optimization software. 


4-23 A construction contractor has undertaken a job 
with 7 major tasks. Some of the tasks can begin at 
any time, but others have predecessors that must be 
completed first. The following table shows those pre- 
decessor task numbers, together with the minimum 
and maximum time (in days) allowed for each task, 
and the total cost that would be associated with ac- 
complishing each task in its minimum and maximum 
times (more time usually saves expense). 


Min. Max. Cost Cost Predecessor 

j Time Time Min. Max. Tasks 

1 6 12 1600 =: 1000 None 

2 8 16 2400 = 1800 None 

3 16 24 2900 2000 2 

4 14 20 1900 =: 1300 1,2 

5 4 16 3800 2000 3} 

6 12) 16 2900 2200 3 

ri 2 12 1300 800 4 


The contractor seeks a way to complete all work in 
40 days at least total cost, assuming that the cost of 
each task is linearly interpolated for times between 
the minimum and maximum. 


(a) Formulate an LP model of this time/cost 
planning problem using the decision variables 


(j=1,....7) 


s; 4 start time of task j (in days) 
i; 4 days to complete task j 


Your model should have an objective function 
summing interpolated cost and main constraints 
to enforce precedence relationships and the 
time limit. 

(b) [| &JEnter and solve your model with the class 
optimization software. 


4-24 Import Books, Incorporated (IBI)!! stocks sev- 
eral thousand titles in its main warehouse. The titles 
can be categorized by sales volume, with i = 1 re- 
quiring a stored inventory of 0 to 20 books, i = 2 re- 
quiring 21 to 40, i = 3 requiring 41 to 100, andi = 4 


“Based on R. J. Paul and R. C. Thomas (1977), “An Integrated Distribution, Warehousing and 
»sentory Control System for Imported Books,” Operational Research Quarterly, 28, 629-640. 
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requiring 101 to 200. The number of titles in cate- 
gory iis b;. Each title is stored in a separate bin, and 
each bin has at most one title. IBI has 500 bins with 
space for up to 100 books and 2000 larger ones with 
space for up to 200 books. The bins for 100 can also 
be subdivided to create either two bins for 40 books 
or three bins for 20. Costs c;; for storing a category 
i title in a size j bin have been estimated by account- 
ing for the material handling cost of accessing the bin 
and the wasted space if a bin is underutilized. Here 
j = 1 refers to bins for 20, j = 2 to bins for 40, j = 3 
to bins for 100, and j = 4 to bins for 200. Formu- 
late an LP model to find a minimum cost alloca- 
tion of titles to bins using the decision variable 
(Sea ao cnt) 


x;; 4 number of titles of category 
i allocated to bins of size j 


4-25 Dairy cows” in most countries calve on a reg- 
ular annual basis. Their milk output varies over the 
year accordingly, with a peak reached a few months 
after calving followed by a decline to almost zero 
in the tenth month. Knowing these facts, farmers 
in an agricultural cooperative are trying to plan 
calving months c = 1,...,12 to make it easier to 
meet seasonal milk demands rg pounds in months 
d = 1,...,12. Any milk produced beyond these 
demands must be sold on the bulk market at b per 
pound below the regular price. The annual cost 
m, of maintaining a cow calving in month c varies 
significantly over the year because low-cost grazing 
is available only in certain seasons. From scientific 
studies the farmers can estimate the yield pyz,- pounds 
in demand month d per cow calving in month c. For- 
mulate an LP model to determine a minimum total 
cost calving schedule using the decision variables 
(Gedi=1,,.....i,, 12) 


x, & number of cows calving in month c 
ya & pounds of excess milk 
produced in demand month d 


4-26 Blue Bell! is planning its monthly produc- 
tion of a particular type of men’s jeans. Demands 
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d; are know for the i = 1,...,75 different fabric 
parts needed to make all the combinations of waist 
and inseam sizes being produced. Such parts are cut 
from fabric laid out on cutting tables in 60 to 70 lay- 
ers. A predefined set of markers (cutting patterns 

m =1,...,350 define how various parts may be cut. 
Each use of marker m yields a; », copies of part i per 
layer and wastes w,, square yards of fabric in areas 
between the usable parts. Formulate an LP model te 
choose a minimum total waste cutting plan using the 
decision variable (m = 1,..., 350; p = 60,..., 70) 


Xm,p & number of times marker pattern m is cut 
in a layup of p layers 


4-27 To assess the impact on the U.S. coal market 
of different pollution control strategies, the Envi- 
ronmental Protection Agency (EPA) wants to de- 
termine, for assumed control regimes, how much 
coal from supplies s; in different mining regions 
i=1,...,24 will be extracted, how much will ther 
be processed into various deliverable coal types m = 
1,...,8, and how much of each deliverable produc 
will be sent to meet consumer demands d,,; in re- 
gions j = 1,..., 113. Demands are expressed in Btu 
with each ton of raw coal mined at i for processing 
into type m yielding a;,, Btu. Including the economic 
burden of pollution controls and transportation, the 
cost per ton of raw coal mined at i for processing int 
type m and use at j can be estimated at Cin,j;. For 
mulate an LP model to determine how coal woule 
be mined, processed, and distributed if the marke 
seeks to minimize total cost. Use the decision var 
able (i= 1)... 247 =157,.3 5.05) = ly) 


Xim,j & tons of coal mined at i for processing 


into m and use at j 


4-28 Quantas Airways Ltd.!° must schedule its hun. 
dreds of reservation salesclerks around the clock * 
have at least r; on duty during each 1-hour perioe 
starting at (24-hour) clock hourt = 0,..., 23. Ashe 
beginning at time t extends for 9 hours with 1 how 
out for lunch in the fourth, fifth, or sixth hours of th: 
shift. Shifts beginning at hour t cost the company 


12Based on L. Killen and M. Keane (1978), “A Linear Programming Model of Seasonality in Milk 
Production,” Journal of the Operational Research Society, 29, 625-631. 
13Based on J. R. Edwards, H. M Wagner, and W. P. Wood (1985), “Blue Bell Trims Its Inventory,” 


Interfaces, 15:1, 34-52. 


14Based on C. Bullard and R. Engelbrecht-Wiggans (1988), “Intelligent Data Compression in a Coal 


Model,” Operations Research, 38, 521-531. 


15Based on A. Gaballa and W. Pearce (1979), “Telephone Sales Manpower Planning at Quantas,” 


Interfaces, 9:3, 1-9. 


per day, including wages and night-hour premiums. 
Formulate an LP model to compute a minimum total 
cost daily shift schedule using the decision variables 
(@=0,...,23;i=74+4,...,f+6) 


x, & number of clerks working a shift starting 
at hour ¢ 


y;; & number of clerks working a shift starting 
at hour ¢ who take lunch during hour i 


4-29 An Indian reservation irrigation project!® must 
decide how much water to release through the gate 
at the top of its main canal in each of the upcoming 
+-hour periods ¢t = 1,...,18. Ideal canal outflows, 
r., are known for each time period, and the total out- 
flow over all 18 periods should equal or exceed the 
sum of these quantities. However, period-to-period 
deviations may be needed to avoid flooding. The 
imitial canal storage is 120 units, and the net effect 
xi releases and outflows should never leave more 
than u units stored after any period. Within these 
imits, managers would like to minimize the total ab- 
solute deviation between desired demands r; and ac- 
tual outflows. Formulate an LP model of this irri- 
zation control problem using the decision variables 
ley, 18) 


x, 4 gate release during period f 
s,; £ amount of water stored in the canal at the 
end of period ¢ 
w, 4 canal outflow during period t 
d; 4 oversatisfaction of demand in period t 
d- A undersatisfaction of demand in period t 
+30 Major shopping mall developer Homart"” is se- 
cting the tenant mix for its next facility. Stores of 
sroduct typesi=1,...,20 are being considered for 
srangement into the new mall’s sectorsj =1,...,5. 
=ach sector will have 150 thousand square feet, and 
= allowance of c; per square foot will be set aside for 
uishing of areas allocated to stores of type 7. From 
=mor experience, Homart can estimate the present 
orth p;; of revenues from a type i store located in 
=ctorj and the required floor space a; (in thousands 


171 


Exercises 


of square feet). They seek a tenant mix that will 
maximize total present worth while having between 
n, and 7; stores of each type i totaling between f > 


and f, thousand square feet, and not exceeding the 
budget b for finishing allowances. Formulate an LP 
model of this tenant mix problem using the decision 
variable 


x;; & number of stores of type i included in sector j 


4-31 Once the configuration of molds is fixed, the 
planning of production of aluminum ingots'® reduces 
to allocating the time of furnaces j = 1,..., 7 among 
alloys i = 1,...,m and ingot sizes s = 1,...,p. 
Yields a;,, of ingots of size s producible from furnace 
j during the entire planning period can be estimated. 
Planning should meet demands d,,, for ingots of al- 
loy i and size s, but this may require misapplying 
some ingots (i.e., trimming some larger ingots sizes s’ 
greater than s to meet demands for size s of the same 
alloy). Misapplications result in a trim-loss cost c;,s',5 
for each ingot cut down from size s’ to s. Managers 
want to find a feasible plan that minimizes the to- 
tal cost of these misapplications. Formulate an LP 
model of this ingot production planning problem us- 
ing decision variables (i = 1,...,m;j = 1,...,; 
Re enya l=) 


Xi; & fraction of time on furnace j dedicated to 
making ingots of alloy i, size s 

Yis’,s & number of ingots of alloy i, size s’, 
misapplied to meet demand for size s 


4-32 S&S operates its large supermarkets!’ on a 
24-hour per day basis using only part-time cashiers 
working shifts of 2 to 5 hours per day. All shifts start 
on the hour. The required number 7, of cashiers 
on duty at a given store is known for (24-clock) 
hours h = 0,...,23, and managers can also esti- 
mate the number of employees b; willing to work 
shifts of lengths 7 = 2,...,5. They seek a shift 
schedule that meets requirements at minimum to- 
tal cashier hours worked. Formulate an LP model 
of this shift scheduling problem using the decision 


‘Based on B. J. Boman and R. W. Hill (1989), “LP Operation Model for On-Demand Canal Systems,” 


nal of Irrigation and Drainage Engineering, 115, 687-700. 


‘7Based on J. C. Bean, C. E. Noon, S. M. Ryan, and G. J. Salton (1988), “Selecting Tenants in a 


»opping Mall,” Interfaces, 18:2, 1-9. 


‘SBased on M. R. Bowers, L. A. Kaplan, and T. L. Hooker (1995), “A Two-Phase Model for Planning 
». Production of Aluminum Ingots,” European Journal of Operational Research, 81, 105-114. 
“Based on E. Melachrinoudis and M. Olafsson (1992), “A Scheduling System for Supermarket 
shiers,’ Computers and Industrial Engineering, 23, 121-124. 
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variable (h = 0,...,23;/=2,...,5) 


x,,, 4. number of cashiers starting 
an /-hour shift at hour h 


Neglect the fact that the numbers working each shift 
must physically be integers. 


4-33 The transmitted gray-scale value gi; of pixels 
i=1l,...,m,j=1,...,n,ina digital space satellite 
photo” is distorted by both the usual random noise 
and a known problem with the video camera that 
effectively multiplies the value for pixel (i,j) by a 
blurring factor b;;. Engineers want to restore the 
image by estimating correct values for each pixel 
in a way that minimizes the total absolute devia- 
tion between predicted (after blurring) and observed 
gray-scale numbers. Formulate an LP model of this 
image restoration problem using decision variables 
(Ghee ear fil aes) 

x;; 4 correct value for pixel (7, j) 

d;, 4 positive deviation of predicted over 

observed value for pixel (i, j) 


A negative deviation of predicted below 
observed value for pixel (7, j) 


d;; 5 


ie 


4-34 The Hanshin expressway”! serves the Osaka— 
Kobe area of Japan. Due to heavy congestion, 
the number of vehicles entering at each ramp j = 

., 38 of the expressway is controlled by a sys- 
tem that reevaluates the situation every 5 minutes 
based on current queue lengths q; at each ramp and 
estimated number of new entry-seeking arrivals d; 
over the next 5 minutes. The system enforces end- 
of-period queue-length limits u; and total traffic ca- 
pacities b; on 500-meter segments 7 = 1,..., 23 of 
the highway. Traffic entering at j affects only down- 
stream segments, and a part may exit before reaching 
any given i. Prior engineering studies have captured 
this behavior in estimated fractions f;; of vehicles en- 
tering at j that persist to consume capacity at i. The 
system seeks the feasible control policy that permits 
the maximum total number of entry-seeking vehicles 
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into the expressway during the next time period. For 
mulate an LP model of this traffic control problem 
using the decision variables (j = 1,..., 38) 


x; & number of vehicles allowed to enter 
at ramp j during the time period 


4-35 Industrial engineers are planning the layout 
of cells i = 1,...,18 in a rectangular manufactur- 
ing facility of x = 1000 by y = 200 feet with a 
6-foot-wide, two-way conveyor system along the y = 
0 boundary. It has already been decided that cells 
will be sequenced along the conveyor in the same 
order as their i subscripts, but the exact geometry of 
the cells remains to be fixed. Analysis of cell load- 
ings has produced lower limits x; and y, on the x 
and y dimensions of each cell. Engineers have alsc 
specified minimum cell perimeters p, as a surrogate 
for area (which would lead to nonlinear optimiza- 
tion). Conveyor traffic will enter and exit at input 
and output stations located at cell x-midpoints along 
the conveyor, and one-way traffic flows from cell : 
to cell j are estimated at f,;. The IEs seck a fea- 
sible design that minimizes total travel (flow times 
distance) on the conveyor. Formulate an LP mode 
of this layout problem using the decision variables 
(Cereals) 


x; 4 left x-coordinate of cell j 
y; & y-depth of cell j 
di 4 positive x-distance between I/O stations 
of cells i and j 
d;, & negative x-distance between I/O stations 
of cells i and j 


4-36 Swift Chemical Company* mines phosphat: 
rock, collects it in inventory piles i = 1,...,8, ame 
blends it to meet contracts with customers k = 
., 25 at profit p;, per ton. The critical measur 
of phosphate content in rock is its BPL. Piles com 
respond to different average BPL contents b; p= 
ton, asset value a; per ton, contract net profit r 


20Based on R. V. Digumarthi, P. Payton, and E. Barrett (1991), “Linear Programming Solutions 
of Problems in Logical Inference and Space-Varient Image Restoration,” Image Understanding and the 


Man-Machine Interface III, SPIE Vol. 1472, 128-136. 


21Based on T. Yoshino, T. Sasaki, and T. Hasegawa (1995), “The Traffic-Control System on the 


Hanshin Expressway,” Interfaces, 25:1, 94-108. 


22Based on A. Langevin, B. Montreuil, and D. Riopel (1994), “Spine Layout Problem,” /nternational 


Journal of Production Research, 32, 429-442. 


23Based on J. M. Reddy (1975), “A Model to Schedule Sales Optimally Blended from Scarce Re- 


sources,” Interfaces, 5:1, 97-107. 


per ton, starting inventory h,, and expected quantity 
g; to arrive from mines vary accordingly. Each con- 
tract includes a minimums, and a maximum 5; num- 
ber of tons to be shipped, along with a minimum P, 

and a maximum p, average BPL content. Managers 
want to schedule blending and sales to maximize 
total profit plus total ending inventory asset value. 
Formulate an LP model of this phosphate planning 
problem using the decision variables (i = 1,..., 8: 

ans) 


xix & tons of rock from pile i included in shipment 
for contract k 


h; & ending inventory in pile i 


4-37 Any convex 3-dimensional object (i.e., a body 
such that the line segment between any two points 
im its volume falls entirely within the volume) with 
flat sides can be described as the set of points (x, y, z) 
satisfying a series of linear constraints™*. For exam- 
ple, a 3- by S- by 9-meter box with one corner at the 
origin can be modeled as 
{Coy 2) 70 <4 = 3,0 y= "5, 0'='z <9} 


Suppose that a stationary object is described in this 
way by constraints 


se Ae, 


end that a link of a robot arm is described at its initial 
osition by the constraints 


ax + biy + cz < d; i=1),.. 


DjX + QV +HZ XS; sl cvang LO, 


The object and the link do not intersect at that initial 
osition, but the link is in motion. Its location is be- 
ag translated from the initial location by growing a 
“ep a > Oin direction (Ax, Ay, Az). Formulate an 
_P in terms of decision variables x, y, z, and a to find 
he smallest step (if any) that will produce a collision 
yetween the object and the link, and indicate how 
se LP would detect the case where no collision will 
cur. 
+38 The principal export of Iceland”? is fish which 
ire very perishable and subject to high day-to-day 
ariation in the size of the catch available for pro- 
sessing. Each day processing begins at any packing 


173 


Exercises 


plant with estimates by of the kilograms of raw fish 
species f = 1,..., 10 that will be available for pro- 
cessing for market m = 1,...,20. The marketable 
volume (in kilograms) of product m in any day is 
at most u,,. Each kilogram of raw fish f processed 
for market m yields 4m kilograms of the final prod- 
uct and produces a gross profit of pm in sales minus 
costs other than labor. Processing fish f into prod- 
uct m requires h,,,,; hours of worker time at stations 
i= 1 (filleting), i = 2 (packing), andi = 3 (freezing). 
A total of g; hours of workers can be obtained for 
workstation i at an average wage of c; per hour. The 
daily plan should maximize total gross profit minus 
wage cost. Formulate an LP model to compute an 
optimal fish processing plan using the decision vari- 


ables'( f=) 2", 10; mt 8 2, 20778 S 1)", 9293) 
Xm & kilograms of raw fish f processed for 
market m 


y; & worker hours at workstation i 


4-39 The US. Air Force (USAF) must procure 


aircraft types i = 1,...,10 and associated muni- 
tion types j = 1,...,25 to meet anticipated sortie 
requirements against target types k = pelesyaln 
weather conditions classes £ = 1,...,8. Targets k 
are assigned a value r;,, and 4» are anticipated under 
weather condition £ in the assumed war scenario. An 
aircraft type i using munitions type j under weather 
conditions ¢ has probability p;;x,¢ of killing a type k 
target each sortie, and a load of b;;,,,¢ munitions j is 
required. A total of s;;,¢ such sorties could be flown 
by each available type i aircraft during the assumed 
war. Currently, there are a; aircraft of type i avail- 
able, and new ones can be procured at $¢; billion per 
unit. Similarly, the current inventory of munitions 
type j is m; and new ones can be procured at $d; bil- 
lion per unit. The USAF wants to buy planes and 
munitions to maximize the total expected value of 
targets it could kill if the assumed war scenario hap- 
pened subject to a current-year procurement bud- 
get of $100 billion. Formulate an LP model of this 
weapons procurement problem using the decision 
vaniablesi\(@— i SOs le es, Dov — Lee 158 
IE Ree 'S) 


*4Based on R. Gallerini and A. Sciomachen (1993), “On Using LP to Collision Detection between 
‘fanipular Arm and Surrounding Obstacles,” European Journal of Operational Research, 63, 343-350. 
Based on P. Jennson (1988), “Daily Production Planning in Fish Processing Firms,” European 


enal of Operational Research, 36, 410-415. 


*6Based on R. J. Might (1987), “Decision Support for Aircraft and Munitions Procurement,” Inter- 


ces 17:5, 55-63. 
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Xij,k,¢ & number of sorties flown by aircraft type i 
with munitions type j against target type 
k under weather conditions class £ 
y; 4 number of new type / aircraft procured 
z; & number of new type j munitions procured 


4-40 North American Van Lines?’ maintains a fleet 
of several thousand truck tractors, each of which is 
owned by one of its contract truckers. Tractors can 
be anywhere in the range i = 0,..., 9 years old. Ev- 
ery 4-week planning period t = 1,..., 13, tractors 
may be purchased new at price p, sold to contractors 
at price s;, traded in to manufacturers at allowance a;, 
and repurchased from contractors at price r;. Only 
new units can be sold or traded as age i = 0, and 
no more total units of any age can be traded in any 
period than are purchased new in the same period. 
The number of tractors in the fleet inventory (i.e., 
with contractors) during each period t must fall be- 
tween a minimum /, and a maximum u, to meet sea- 
sonal moving demands without accumulating excess 
capacity. Managers wish to find a plan that max- 
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imizes their net total income from buying, selling. 
trading, and repurchasing trucks. Formulate an LP 
model of this fleet management problem using the 


decision variables (i = 0,...,9;f=1,...,3) 
w, 4 number of new tractors purchased in 
period t 
x;, 4 number of tractors of age i sold to contractor 
in period t 
yiz & number of tractors of age i traded in 
period ¢ 


Zi. & number of tractors of age i repurchased from 
contractors in period t 
fi: & number of tractors of age 7 in the fleet 
at the beginning of period ¢ 
Assume that vehicles enter and leave the fleet only 
through sales to and repurchases from contractors 
Also assume that the fleet ages 1 year as inventor 
is passed from period 13 to period 1 and that ne 
9-year-old vehicles can be carried over to the new 
year. 


Chvatal, VaSek (1983), Linear Programming, W.B 
Freeman, San Francisco. 


27Based on D. Avrmovich, T. M. Cook, G. D. Langston, and F. Sutherland (1982), “A Decision 
Support System for Fleet Management: A Linear Programming Approach,” Interfaces, 12:3, 1-9. 
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Simplex Search 
for Linear Programming 


Having sampled in Chapter 4 some of the enormous variety of linear programming 
models, it is time to focus on the powerful algorithms that make them tractable. In 
this chapter we develop a special form of improving search called the simplex method. 
Simplex is the most widely used of all optimization algorithms, although the newer 
interior-point methods of Chapter 6 are proving strong competitors. Both exploit the 
special properties of linear programs to produce highly efficient algorithms capable 
of globally optimizing huge models. Our treatment assumes reader familiarity with 
the search fundamentals in Chapter 3. 


5. | LP OPTIMAL SOLUTIONS AND STANDARD FORM 


We begin our investigation of linear programming algorithms with some observations 
and conventions that enormously simplify the task of designing efficient computa- 
tional procedures. 


EXAMPLE 5.1: TOP BRASS TROPHY 


As usual, it will help to have a tiny example at hand. We illustrate with the case 
of the (fictional) Top Brass Trophy Company, which makes large championship tro- 
phies for youth athletic leagues. At the moment they are planning production for 
fall sports: football and soccer. Each football trophy has a wood base, an engraved 
plaque, a large brass football on top, and returns $12 in profit. Soccer trophies are 
similar except that a brass soccer ball is on top, and the unit profit is only $9. Since 
the football has an asymmetric shape, its base requires 4 board feet of wood; the 
soccer base requires only 2 board feet. At the moment there are 1000 brass footballs 
in stock, 1500 soccer balls, 1750 plaques, and 4800 board feet of wood. What trophies 
should be produced from these supplies to maximize total profit assuming that all 
that are made can be sold? 
The decisions to be made in this problem are 


x, 4 number of football trophies to produce 


x2 4 number of soccer trophies to produce 


ils 
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In terms of these decision variables, we can model the problem 


max 12x; + 9x2 (profit) 
Site= hy < 1000 (footballs) 
x2 < 1500 (soccer balls) 


xi+x.  <1750 (plaques) Cm 


4x; +2x. <4800 (wood) 
x1, X2 >0 


The objective seeks to maximize total profit, and the main constraints enforce limits 
on footballs, soccer balls, plaques, and wood, respectively. 

Figure 5.1 solves the problem graphically. An optimal solution occurs at xf = 
650 and x} = 1100 with a total profit of $17,700. 


x, < 1000 (x, > 0) 


NK 
x. 
ro) 
@ 
es 
o 


N} 
2g x) $1500 (x, 2 0) 


optimal x* = (650, 1100) 


231 


FIGURE 5.1 Graphical Solution of the Top Brass Trophy 
Example 


Model (5.1) is a linear program (definition | 4.1 |) because the single objective 
function and all constraints involve only weighted sums of the decision variables 
Also, all variables are continuous. 


Global Optima in Linear Programs 


By applying some of the observations in Chapter 3, we can begin to appreciat: 
the elegant model tractability implied by the characteristics required in definitic= 


[4]. Linear objectives are unimodal (principle |3.27|). Feasible sets defined & 


5.1 LP Optimal Solutions and Standard Form LT 


continuous variables and linear constraints are convex (principle | 3.32 |). Thus linear 


programs have all the properties most convenient for improving search, and principle 


applies. 


Figure 5.1 illustrates graphically the Top Brass example. The unique local 


maximum is the global maximum. Notice how property simplifies the task of 
designing LP algorithms. Most implementations of improving search Algorithm 3A 
will produce local optima (principle [317)), and we now know that this is enough 
to obtain globally optimal solutions for linear programs. All our attention can be 
focused on making improving search efficient. 


Interior, Boundary, and Extreme Points 


One useful tool is a classification of the points in an LP feasible region. Not all have 
equal interest. 

We first distinguish between points in the interior of the feasible region versus 
those on the boundary. Precise definitions follow from the fact that the boundary of 
an LP’s feasible set is characterized by active inequality constraints (those satisfied 
as equality). 


active. 


Figure 5.2 labels some specific points in the Top Brass Trophy feasible set. Each 
of the inequalities has some feasible points that satisfy it as a strict inequality, and 
solution x is interior because none of the inequalities is active. All of x, x, 

., x are boundary points because at least one inequality is active at each. For 
example, inequalities x; < 1000 and x2 > 0 are satisfied as equalities at solution 
x = (1000, 0), and x; > 0 is active atx. Solutions x® and x constitute neither 
boundary nor interior points because both are infeasible. 

Extreme points (also called corner points) such as x, ..., x) of Figure 5.2 
are special boundary points so-named because they “stick out.” To obtain a formal 
definition, we must recall that the feasible region of a linear program is a convex 
set and that convex means (definition ) that the line segment between any two 
feasible points lies entirely within the set. Ifa point is to form a “corner” of a feasible 
set, it cannot fall in the middle of any such line segment. 


53 Extrem oii ts of convex sets are those at do not lie within: the line 


Notice that pointsx®, ...,x©) in Figure 5.2 all fulfill this requirement. They can 
form endpoints of line segments within the set, but never midpoints. Contrast with 
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x. 


2 


FIGURE 5.2 Interior, Boundary and Extreme Points of 
the Top Brass Example 


nonextreme feasible solutions x© = (0,500) and x” = (500, 500). The line segment 
from x to x® passes through x, and the segment from x to x = (501,500) 
includes x". 


: ze 


Classify the labeled solutions as interior, boundary, and/or extreme points of the 
following LP feasible region: 


Analysis: Points (a), (c), and (d) are both extreme points and boundary points, be- 
cause they do not lie along the line segment between any two other feasible points 


5.1 LP Optimal Solutions and Standard Form 


Points (b) and (e) are boundary points that are not extreme; each makes at least 
one constraint active, but these points do fall within a line segment joining others. 
Point (f) is interior because no constraint is active. Point (g) is neither interior nor 
boundary (nor extreme) because it is infeasible. 


Optimal Points in Linear Programs 


Linear programs achieve their optimal solutions on the boundary (except in trivial 
cases with all feasible solutions having the same objective value). 


5.4_| Every optimal solution to a typical linear pr: ogram will be a pee 
- point of its feasible region. 


For example, the optimal solution to our Top Brass example (see Figure 5.1) occurs 
at boundary (and extreme-point) solution x* = (650, 1100). 

To gain some intuition about why LP optima usually occur on the boundary, 
consider the case where all constraints are inequalities. Observe that we can make at 
least a small move in any direction from an interior point without losing feasibility. 
Also, with a (nonzero) linear objective function, the coefficient vector Ax = c is 
always an improving direction for a maximize problem because it is the gradient 


(principle [3.23]), and Ax = —c improves for a minimize problem. Thus there is 
always an improving feasible move at interior points of a feasible set defined by 
inequalities unless c = 0 and all solutions cost the same. Matters become more 
complex with equality constraints present, but the result is the same for every typical 
LP model. No interior point can be optimal. 

What about unique optimal solutions? Intuition suggests that if a feasible point 
is to be the only optimal solution to a linear program, it must somehow “stick out” 
farther than other feasible points. This is true. 


5.5_| Ifa linear program has a a unique optimal solution, that optimum must 
occur at an extreme point of the feasible region. 


To see exactly why, try to think of an objective function making non-extreme 
boundary solution x the unique optimum in Figure 5.2. The closest we can come is 
min x, 


which has x© optimal but not uniquely so. 
We can never make nonextreme boundary solutions uniquely optimal because 
such points always lie within a line segment joining two other feasible points (def- 


inition | 5.3|). Thus they can be expressed as a weighted average of the endpoints 


of such a segment (principle | 3.31 |), and their objective value is the same weighted 
average of those for the two endpoints. Either all points of the segment have equal 
objective value, or one of the endpoints is superior to all the rest. 

Continuing in this way, we can see that no collection of nonextreme boundary 
points can contain all the optimal solutions to a linear program. The collection could 
always be expanded, with the endpoints of line segments containing known members. 

We would inevitably have to come upon an extreme-point optimum. 
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56 | Ifa linear program has any optimal solution, it has one at an extreme 
point of its feasible region. Se 

Of course, the LP may be infeasible or unbounded, and there may be non-extreme 

alternative optimal solutions along the boundary. But if a linear program has any 

optimal solutions at all, at least one will occur at an extreme point. 

Principle is fundamental because of the flexibility it provides in designing 
improving search algorithms for linear programming. Knowing that there will be an 
optimal solution at an extreme point if there is any optimal solution at all, we are 
free to restrict our search to extreme-point solutions. The simplex algorithm, which 
is the main topic of this chapter, does exactly that. 


SAMPLE EXERCISE 5.2: IDENTIFYING OPTIMAL POINTS —_ 


Indicate which of the labeled points in Sample Exercise 5.1 can be optimal or uniquely 
optimal for any objective function. 


Analysis: Following principle [5 extreme points (a), (c), and (d) can be optimal 
or uniquely optimal. Boundary points (b) and (e) can also be optimal. Still, nei- 
ther can be uniquely optimal because any objective making either optimal will also 
have extreme-point optimal solutions. Interior point (f) cannot be optimal for any 
nonzero objective function; feasible improvement is always possible. Point (g) is not 
even feasible. 


LP Standard Form 


A model can be a linear program even if variables are subject to a variety of (con- 
tinuous) variable-type constraints, main constraints are mixtures of inequalities and 
equalities, and expressions are nested through several levels of parentheses on both 
sides of >, <, and = signs. Still, it will be much easier to discuss solution methods if 
we settle on an LP standard form. 


side, and any constant term (possibly zero) appears on the right-hand side. 


Converting Inequalities to Nonnegativities with Slack Variables 


We will clearly have to do some rearranging to fit every “raw” linear program into 
standard form. For example, Top Brass model (5.1) includes inequalities among 


its main constraints. Standard form allows inequalities only in the form of 
nonnegativity variable-type constraints. 

To accomplish the needed transformation we introduce new slack variables 
in each main inequality that consumes the difference between left- and right-hane 
sides. 


5.1. LP Optimal Solutions and Standard Form 


- Main inequality constrai 
ed into nonnegativities by addin 

ables i in every such < inequality and subtracting such slack os in every — 
_ mam =. 


Applying construction | 5 BF 8 | to the 4 main inequalities of Top Brass model (5.1), 
we add slacks x3,..., x6. The result is the standard-form model 


max 12x, + 9x» 

s.t. xX +.x3 = 1000 
x2 +x4 = 1500 
Xi + X2 +xs5 = 1750 
4x, +2x. +x6 = 4800 
Mie Kan X35 X4e W655 Nore 0 


(5.2) 


Notice that a different slack was used in each constraint, nonnegativity limits apply 
to each slack variable, and that the slacks do not appear in the objective function. 
All slack variables carry plus signs in this example (compare with Sample Exercise 
5.3 below) because all the modified inequalities were of the < form. 

Even though our standard form now has 6 variables versus an original 2, we 
have not really changed the model. Labels in Figure 5.1 show that each original 
inequality constraint corresponds to some slack’s nonnegativity in standard form. 
For example, the final main constraint 


Ax, + 2x2 < 4800 
becomes nonnegativity on slack variable x5. Under standard-form equality 
4x, + 2x2 + x6 = 4800 
the inequality holds exactly when 
X6 = 4800 — 4x, — 2x2 > 0 


Also, with their coefficients in the objective function =0, slack variables have no 
impact on cost. 

Why bother with this modified version of a perfectly good linear program? To 
see, we must look back to the development of feasible move directions in Section 3.5. 
Most of the complexity in dealing with feasible directions relates to keeping track of 
active inequality constraints (ones satisfied as equalities). Equality constraints are 
always active, but inequalities may be active at one moment in a search and inactive 
at the next. Introducing slack variables as in construction does not eliminate 
any inequalities, but it does convert them to the simplest possible form which will 
simplify analysis. 


SAMPLE EXERCISE 5.3: INTRODUCING SLACK VARIABLE 


Introduce slack variables and simplify to place each of the following linear programs 
in standard form. 
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(a) min 9w, + 6w2 


s.t. 2w, + Ww > 10 
W4 i < 50 
Ww, + WwW = 40 
100 > w,; +2w2 > 15 
W1,W2 > 0 


(b) max 15(2x; + 8x2) — 4x3 
s.t. 2(10 — x1) +2 + 5(9 — x3) = 10 
xX, + 2x3 < x3 
2x2 + 18x3 = 50 
Migs = 0 


Modeling: 


(a) We introduce slack variables w3, w4, ws, We as in construction to convert 
the 4 main inequalities to nonnegativities. The result is the standard-form linear 
program 

min 9w,+6w2 


st. 2w, +wo —w3 = 10 
W1 + w4 = 50 
W141 +W2 = 40 
Ww, + 2w2 +ws = 100 
wit+2w. —we =15 


W1,W2, W3, W4, W5, Wo = 0 
Notice that the last two main constraints, which were written together in the origi- 
nal model, are separated in standard form. Slacks are added in < inequalities and 
subtracted in > forms. No slack is required in equality constraints. 


(b) We begin by simplifying to collect variable terms on the left-hand side and con- 


stants on the right [(3) of definition ]. 
max 30x; + 120x2 — 4x3 


s.t. —2x, +x. —-—5x3 > S55 
Xx +3 Soe) 
2x2 + 18x3 == 5() 


X1,X2,%3 2 0 
Now slacks are introduced according to construction to complete the standard 
form 
max 30x, +120x. — 4x3 


s.t. —2x1 + X2 — 5x3 —X4 = —55 
xy +.X3 =a == eee) 
2x2 + 18x3 = () 


X1,.X2,X3,%4,X5 => 0 


Converting Nonpositive and Unrestricted Variables to Nonegative 
Like most applied linear programs, the Top Brass Trophy example employs only 


nonnegative decision variables. Condition (2) of standard format | 5.7) is fulfilled 
automatically. 


5.1. LP Optimal Solutions and Standard Form 


Still, LPs do occasionally involve variables such as net income or temperature 
that can feasibly take on negative values. Such variables may be nonpositive (i.c., 
subject to < 0 variable-type constraints) or unrestricted by sign. The latter are often 
designated URS, meaning “unrestricted sign.” 

We can convert linear programs with nonpositive and unrestricted variables to 
standard form simply by changes of variables. For nonpositive variables, the change 
substitutes the negative. 


5.9 | Nonpositive variables in linear programs c can be eliminated by substitut- 
ing new vai iables equal to their negatives. 


For example, in a model with original decision variables x1, ..., x19 and nonposi- 
tive-type restriction 


x7 <0 
we would introduce the new variable 
x4. = —X7 


and substitute —x;; everywhere that x7 appears. In particular, the nonpositivity 
constraint 


x7 <0 becomes —-x1, <0 or xy >0 


The handling of unrestricted variables is slightly less obvious. How can non- 
negative variables model a quantity that can take on any sign? The answer is to 
introduce two new nonnegative variables and consider their difference. 


5.10 | restricted (or URS) variables in linear programs can be eliminated by 
_ Substituting the difference of two new nonnegative variables. 


For example, a model with variables y;, ..., y7 and 
VM URS 


can be placed in standard form by introducing two new variables yg, yo > 0. Every- 
where y; appears in the given model we then substitute 


Nile) ie p) 


SAMPLE EXERCISE 5.4: CONVERTING NONPOSITIVE A AND D UNRESTRICTED 
VARIABLES © 


Make suitable variable changes to convert the dutseiat linear program to standard 
form: 
min —9w, + 4w2 + 16w3 — 1lw4 
st. wy + Ww. +w3+ Wa = 100 
3w1 — Wo + 6w3 — 2wa = 200 
Ww, > 0, w2 <0 
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Modeling: Variable w is already nonnegative, but we must substitute for nonpositive 
variable w2 and for URS variables w3 and w4 which are subject to no variable-type 


constraints. Following rule and , we employ 
W2 = —W5 
W3 = W6—-W7 
W4 = Wg — Wo 


After simplification the resulting standard-form linear program is 


min —9w, —4ws + l6we — 16w7 — 11we + 11W9 

st. Wy —W5 + Wo —W7 + Wes — Wo = 100 
3w, + ws + 6wo — 6w7 — 2wg + 2wo = 200 
W1,W5, W6, W7, Wg, Wo = 0 


Notice that w2, w3, and w4 have been eliminated from the model completely. 


Standard Notation for LPs 


Once a linear program has been placed in standard form, its key elements are so 
neatly sorted out that we can begin to think of the model as a collection of coefficients. 
Widely used notation gives all these elements standard names. 


S11 L. Standard 


yo 2 jth decision variable 

y = is & cost or objective function coefficient Ox, 

(G2 A constraint coefficient of x; in the ith main constraint oe 
4 A A right-hand-side (or RHS) constant term of main constraint L 

-m & number of main (equality) constraints _ a 

n © number of decision variables = 


tation for linear prog ans’ is 


Then, LP standard form of every linear program has the generic format 


min (or max) Ss CjXj 
j=l 


S.t. be Qi jXji = b; for alli = 1; 2, sreeg FTL 
j=l 
x; 20 fOnvally =k 2 eee i 


It will often be convenient to write standard-form linear programs in an even 
more compact way using notions of vectors and matrices. Primer 3 reviews some of 
the main ideas of matrix arithmetic for those who need it. 


— | In the usual matrix notation, LP standard form is 
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_ Matrices are 2 dimensional arrays of aunbers: with the dimension of the array _ 
being described i in terms of its number of rows and coms, For example, 


are 2 by 3 matrices ae TOWS and 3 columns), ‘See Primer 1 for the related 1- 
dimensional notion of Vectors. 


cated by so eeeonding lowercase 
italic symbols (a;;. 72.6, a3, 3) h pt for the row index and one for 
the column. Thus matrix Q above has qi. = 0. and q2.3 = 3. 
_ Just as with ve ctors, matrices of like dimension are added, subtracted, and. 


0.6 —0.42 
0.6 


Matric P and A can Be multiplied as D = PA if the number of columns in P_ 
as the number of rows in A. ‘Then the i, J ee of the e result is 


& -1 6 St 2a, 12 
2a 9 ne mo( ndel8 5%) : 


ed for the P and A — because the number of columns of ‘A does 

not equal the number of rows of P. Even if matrices multiplication 1 in 
both orders, the results can be different. 

of a matrix bya vector is defined as if the vector were arow 


and A is as above, 
vA 2c —1, a(3 


matrix A. In the second computation x plays the role fa 
because it postmultiplies A. 


Cost or objective function vector ¢ presents all objective function coefficients cj, 
constraint matrix A displays all main constraint coefficients a;;, and right-hand-side 
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(RHS) vector b shows constant terms b; of the constraints. As usual, x is the vector 
of decision variables ;. 

We can illustrate with the Top Brass Trophy model and its m = 4 main con- 
straints. Instandard form the Top Brass decision vector has the form x = (x1, x2, ..., X- 
with n = 6 components. Corresponding coefficient arrays are n-vector c, m by n ma- 
trix A, and m-vector b: 


¢ =(12 9" 0-0-0. 0 ) 


rea har 1000 
Od: 0 eon0 1500 
* = [ge ai boneobetenG b=) 4750 
4. DOL OUROHIOr af 4800 


SAMPLE EXERCISE 5.5: USING STANDARD LP MATRIX NOTATION : 
Return to the standard-form linear program of Sample Exercise 5.3(b). Identify the 
m,n, A, b, and ¢ of standard matrix representation ee * 


Modeling: The standard form model of Sample Exercise 5.3(b) has m = 3 main 
constraints and n = 5 variables. Corresponding coefficient arrays are 


a= Sonia O20) 


—2 Ss) ll 0 =55) 
A= 1 0 1 0 1 b= 0 
0 2 Als 0 0 50 


& 2 EXTREME-POINT SEARCH AND BASIC SOLUTIONS 


One can fashion efficient algorithms for linear programs that pass through the interior 
of the feasible set (see Chapter 6). Still, since there is an extreme-point optimal 


solution if there is any at all (property [5.6]), we are free to limit our search to 
extreme points of an LP’s feasible region. 

Simplex is just such an extreme-point search algorithm. In this section we 
develop some underlying concepts that lead to the full algorithm in Section 5.3. 


Determining Extreme Points with Active Constraints 


To begin, we need a more convenient characterization of extreme points. Extreme 
points are boundary points, and every part of the boundary of an LP feasible space 
is formed by one or more active constraints. What makes extreme-point solutions 
special is that enough constraints are active to determine a point completely. 


For example, extreme-point solution x in Top Brass Figure 5.2 is determined 
completely by the two constraints active there, x; > 0 and x2 < 1500. Contrast this 


5.2 Extreme-Point Search and Basic Solutions 


with point x, where only x; > 0 is active. This is not enough to determine a point 
because many solutions have x; = 0. 

To see more fully how active inequalities relate to extreme points, we need 
more dimensions. Figure 5.3 depicts the search of a 3-dimensional case maximizing 
x3. The boundary area where each main inequality is active now forms one of the 
2-dimensional faces labeled A-L. Extreme points are determined as in principle 


by collections of three active constraints. 


FiGureE 5.3. Adjacent Extreme Point Search to Max x, 
in Three Dimensions 


Inequalities J andJ, plus nonnegativity x3 > 0, determine x), Notice, however, 
that some extreme points can be determined in several ways. For example, any 3 of 
the inequalities for surfaces C, D, G, and J determine the extreme point labeled x®), 


SAMPLE EXERCISE 5.6: DETERMINING EXTREME POINTS 


List all sets of 3 active constraints determining points x, x“ and x in Figure 5.3. 


Analysis: Constraints B, C, G, H, and J are all active at x). Any 3 of these 5 
determine the point. Solution x is determined uniquely by constraints A, B, and 
C. Extreme point x® is determined uniquely by inequalities A and C, plus x2 > 0. 


Adjacent Extreme Points and Edges 


Improving search algorithms move from one solution to another nearby. We can 
define neighboring or adjacent extreme points in terms of their determining sets of 
active constraints. 
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Again Figure 5.3 illustrates. Extreme points x and x) are adjacent because 
the first is determined by active inequalities G, J, and J, while D, G, and J provide a 
determining set for the second. These two lists have all but one member in common. 
Contrast with x and x“), which are not adjacent. Point x is determined by 
active inequalities A, B, and C. Of these, only C is also active at x: no choice of a 


Adjacent extreme points are joined by an edge determined by the active con- 
straints the extreme points have in common. For example, inequalities for surfaces 
G, I, and J determine the extreme point labeled x. Keeping just G and J active 
produces the edge joining x) and x”. Similarly, if J and G are kept active, we obtain 
the edge between adjacent extreme points x"? and x®). 


—2x, + 3x2. <6 (5.3) 
=a ted, Sell (5.4) 
x, >0 (5.5) 

x. >0 (5.6) 


(a) Sketch the feasible region and identify points x = (0,0), x = (0,1), x® = 
(3,4), and x® = (4, 0). 


(b) Determine which pairs of those points are adjacent extreme points. 
(c) Determine which pairs of the points are joined by an edge. 
Analysis: 


(a) The feasible region is as follows: 


5.2 Extreme-Point Search and Basic Solutions 


(b) Applying principle [5.13 |, extreme points must be determined by active con- 


straints. Solutions x, x, and x are thus extreme points determined by active 
constraints (5.5)-(5.6), (5.4)-(5.5), and (5.3)-(5.4), respectively. Point x is not an 
extreme point. 


if they have one active constraint in common. Thus x and x“) are adjacent, as are 
x and x®), but not x and x®. 


(c) Each adjacent pair of part (b) is joined by an edge determined by their common 
active constraint. For example, x“ and x are joined by the edge along line 
—X, +x. =1 
Although x® is not an extreme point, the edge defined by line 
x, = 0 
joins it to x. 


Basic Solutions 


We saw in Section 5.1 how LP standard form encodes every model as 
n 
min (or max) ys CjXj 
j=l 
n 
s.t. Aj jx; = D; for alli=1,2,...,m 
j=l 
xj = 0 torall j= 28 tnt 


One effect is to reduce all inequalities to nonnegativity constraints. 


It follows from principle that extreme-point solutions to linear programs 
in standard form are determined by sets of active nonnegativity constraints. Enough 
variables must be fixed to =0 (making the corresponding nonnegativity constraints 
active) to uniquely determine all other components of a solution. 

Basic solutions are produced in just this way. 


We can illustrate with the standard form of our Top Brass example (Sec- 
tion 5.1): 


max 12x, +9x2 


s.t. + x4 +x = 1000 
+ X2 +Xx4 = 1500 

+x + % +Xx5 = 1750 G7) 
+4x, +2x2 +x5 = 4800 


Ki, 2, X53, Xa, Xs, Xe. = 0 
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Figure 5.4 again shows the feasible set, with constraints labeled by the applicable 
nonnegativity constraint of standard form. 

One basic solution is obtained by choosing x1, x2, x3, x4 to be basic and xs, x6 
to be nonbasic. Fixing x; = x5 = 0 leaves the equality system 


+ x1 +%3 = 1000 

a aaa = 1500 
+n + x +(0) = 1750 fe 
+4x, +2x, + (0) = 4800 


The unique solution is x, = 650, x. = 1100, x3 = 350, and x, = 400. Thus the full 
basic solution is x = (650, 1100, 350, 400, 0, 0). 

Notice that this standard-form solution corresponds to extreme point x®) in 
Figure 5.4. This should come as no surprise, because x is defined by making active 
the inequalities corresponding to x5 > 0 and x6 > 0. That is, the point is defined by 
setting x5 = x, = 0. 


SAMPLE EXERCISE 5.8: COMPUTING BASIC SOLUTIONS 
Suppose that a linear program in standard form has the constraints 


4x, -— % + 3 =1 
3x1 + 2x —2x3 =8 
Lip copeka=O 


Compute the basic solution corresponding to x; and x2 basic. 


Analysis: The only nonbasic variable will be x3. Setting it to zero as in construction 


gives 


4x, — x» + (0) =1 
3x1 +2x, -—2(0) = 8 


The remaining 2 equations in 2 unknowns have a unique solution x; = 1, x. = 3. 
Thus the full basic solution is x = (1, 3, 0). 


Existence of Basic Solutions 


It is tempting to believe that we can form basic solutions by setting any collection of 
nonbasic variables to =0 (i.e., by making any collection of nonnegativity constraints 
active). Not so! For example, fixing only x, = 0 in the Top Brass standard-form 
model leaves the equation system 


“Me +3 = 1000 

+ Xx + (0) = 1500 
= Xa ale +Xx5 = 1750 
+4x, 4+2x2 +x6 = 4800 


There remain S unknowns in only 4 equations. Geometrically, this simply reflects 
the fact that only making active the inequality corresponding to x4 > 0 in Figure 5.4 
does not fully determine a point. 


5.2 Extreme-Point Search and Basic Solutions 


FiGurRE 5.4 Basic Solutions of the Top Brass Example 


We are no better off if we fix x. = x4 = 0. The resulting system 


+ xy +.x3 = 1000 

+ (0) + (0) = 1500 (5.9) 
+ x, +(0) +X5 = 1750 ; 
+4x, +2(0) +x— = 4800 


now has 4 equations in 4 unknowns, but it has no solutions as all (look carefully 
at the second equation). This establishes algebraically what is apparent in Figure 
5.4. Making active the inequalities corresponding to x. > 0 and x4 > 0 does not 
determine a point. 

To characterize when basic solutions do exist, we must draw on the algebra of 
systems of simultaneous equations. Primer 4 provides a quick review. One condition 
assuring a system with a unique solution gives basic solutions their name: 


5.17| A basic solution exists if and only i if the columns of equality constraints — 
corresponding to basic variables forma basis, that is, a gee a 
independent collection. . _ 


Systems (5.8) and (5.9) illustrate the possibilities. We can verify that the con- 
straint columns for the basic variables x1, x2, x3, and x4 in (5.8) form a basis by 
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checking that the corresponding matrix is nonsingular, that is, 


It follows that the equations have a unique solution. 

On the other hand, the columns for basic variables x1, x3, x5, and x¢ in (5.9) do 
not produce a unique solution because they are linearly dependent; the column for 
x, is easily expressed as 


1 
0 
ia hake 
4 


o°oor 
of oss 


The foiteabitg are the constraints of a linear program in cuantiard form: 


4x4 = 8x2 08) 15 
xX — 2x2 = 10 
X1,X2,x3 = 0 
Check whether basic solutions exist for each of the following possible sets of basic 
variables: (a) x1, x2; (b) x1, x33 (C) x1; (d) x2, x3; (€) X1, X2, x3. 
Analysis: 


(a) Column vectors for x; and x2 form a linearly dependent set because 


( : ) = -3/ e ) or equivalently, det ( ; = ) =0 


Thus basic solutions do not exist. 


(b) Basic solutions do exist because column vectors for x; and x3 form a basis. One 
way to check is to verify that the corresponding matrix is nonsingular, that is 


4 -1 
det ( 1 0 ) lee") 
(c) Column vector (4, 1) for x; is linearly independent because it isnonzero. However. 


we have already seen in part (b) that a larger linearly independent sct of columns is 
possible. Thus x; alone does not determine a basic solution. 


(d) Basic solutions do exist because column vectors for x2 and x3 form a basis. One 


proof is 
aet ( cae )=-2#0 


(e) Columns for this set of variables cannot form a basis because no more than two 
2-vectors can be linearly independent. Thus basic solutions do not exist. 
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solution, no solutions, or an infini 
m= instances _ 


ee —6y; — yo + Ily3 =—2 —~67, -Z2+¢1lz3= 1 
have unique solution x = (1,0, —2), no solution nd infinitely many solutions 
Z. reply. 2 oe 

Whether a system falls within ie aE un 

. tirely on the variable coefficient structure of i 

_ above continues to have a unique solution if t ht-hand side (17, 1, 24) is 
_teplaced by any other 3-vector. Notice that the same is not true of the two 
_ nonunique cases; the y and z systems above have identical coefficients on the: 
- left-hand side. 
A square matrix is singular if its determina: 0 and poacinigulay othe 
_ wise. In these terms, systems of m linear equations in m unknowns have unique 
solutions if and only if the corresponding matrix of left-hand-side variable coe 
rminant tot a a Square matrix Da is ie Sar a 


td) 2 = dy 


and D, the matrix obtained from D = deleting row | and column j. Thus oe 
stem above has a unique soluti nd ihe pies do not, because corre- 


6 =1 


are fonsingulat and sin 


: det(N) _ na 


Cee 
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Aor the various variable cient = . 
aS ore oe : 


On the other hand, corresponding y nd z sy 
solutions because columns fs, s@7 8°) are lin 


Basic Feasible Solutions and Extreme Points 


Table 5.1 enumerates all possible choices of 2 nonbasic and 4 basic variables in the 
4 equations of the Top Brass standard form. In two cases there is no basic solution 


because (principle |5.17 |) the equality constraint columns corresponding to basic 
variables are linearly dependent. 

All other combinations produce basic solutions. Notice, however, that nothing 
in the construction of basic solutions (definition | 5.16 


) guarantees feasibility. Some 


5.2 Extreme-Point Search and Basic Solutions 


TABLE 5.1 Basic Solutions of the Top Brass Example 


Active Basic Basic Solution Point in 
Constraints Variables Solution Status Fig. 5.4 
x, >0,x>0 X3, X4, Xs, Xp x = (0, 0, 1000, 1500, 1750, 4800) Feasible x(0) 
X, >0,x320 Xa, X4y 5, XG None Dependent — 
X,20,x%420 —Xp,%5,X5,X§ x =(0, 1500, 1000, 0, 250, 1800) Feasible x) 
x, 20, x5>0 Xo, X3, X4, Xe x = (0, 1750, 1000, —250, 0, 1300) Infeasible x(9) 
X,20,%6 50  X4,%3,X4,X5 x = (0, 2400, 1000, -900, -650, 0) Infeasible x(7) 
X>0,%3>0  %1,X4,%5,X%§ x = (1000, 0, 0, 1500, 750, 800) Feasible x() 
Xz >0,x4>0 X71, X3, Xs, XG None Dependent — 

X_ > 0, x5 >0 X1,.%3, X4, X6 x = (1750, 0, -750, -1500, 0, -2200) Infeasible x(12) 
X_>0,%6>0  x4,%3,%4,%5 x = (1200, 0, -200, 1500, 550, 0) Infeasible x(l) 
x3 >0,x4>0 X41, X7, Xs, XG x = (1000, 1500, 0, 0, -750, -2200) Infeasible x0) 
%3>0,%5>0  —X4,%4,%4,%5 x = (1000, 750, 0, 750, 0, -700) Infeasible x(10) 
x3 >0,%5>0 X15 Xqy X4y X5 x = (1000, 400, 0, 1100, 350, 0) Feasible x) 
x420,%3>0  X4,Xy,%3,X%% x = (250, 1500, 750, 0, 0, 800) Feasible x) 
X42 0,%5>0 X1, Xo, X3, Xs x = (450, 1500, 550, 0, —200, 0) Infeasible x() 
x5>0,%620  —X4,X2,%3,X4 x = (650, 1100, 350, 400, 0, 0) Feasible x) 


of the solutions in Table 5.1 violate nonnegativity constraints. These correspond 
geometrically to points where active inequalities determine a solution falling outside 
the feasible region. For example, choosing x3 and x4 nonbasic yields point x in 
Figure 5.4, which has negative components for standard-form slack variables of the 
two constraints it violates. 

Our interest is in basic feasible solutions. 


5.18 | A basic feasible solution to a linear program in standard form is a basic 
solution that satisfies all nonnegativity constraints. 


Comparison of Table 5.1 and Figure 5.4 will show why. The six basic feasible solutions 
are exactly the six extreme points x, ...,x© of the feasible region. 
This is no accident. We have seen how extreme points are feasible solutions 


determined by collections of active constraints (property ). For linear programs 
in standard form, which have only nonnegativity-form inequalities, this means that 
extreme points are determined by basic solutions. It follows that the extreme-point 
solutions are the basic ones that are feasible. 


a the extienie poin solutions: of its feasible region. 


SAMPLE EXERCISE 5. 10: IDENTIFYING Basic FEASIBL E g SOLUTIONS 


The constraint set of a standard-form linear program is defined by the following 
constraints: 


=X) Sh = X3 
+X1 +X4 


0 
2 
3 


+ 
S 
i | 


keen, poses O) 
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(a) Assuming that variables x3, x4, and x5 are slack variables added to produce 
standard form, graph the feasible region in the original variables x; and x2. 


(b) Compute the basic solutions corresponding to each of the following sets of basic 
variables, and determine which are basic feasible solutions: B; = {x3, x4, x5}, Bo = 
{X2, X4, x5}, B3 = (x1, x2, %5}, Ba = (01, X2, X4}, Bs = (x1, x3, X5}. 


(c) Verify that each basic feasible solution of part (b) corresponds to an extreme point 
in the graph and that each infeasible basic solution corresponds to a point outside 
the feasible region determined by the intersection of constraints. 


Analysis: 


(a) Original constraints would have been 


(b) Fixing nonbasics x; = x2 = 0 for basic set B,, the resulting equation system is 


=O) 
+ (0) 


Solving produces basic solution x = (0, 0,0, 2,3). Results for other basic sets are 


derived in the same way: 
Bo 
B3 
Ba 
Bs 


Notice that different basic sets (here B; and Bz) can produce the same basic solution. 
Under definition , only x, x®, and x®) are basic feasible. The others 


-xX, +x >=0 
x 2) 
xX, <3 

X1,X2 > 0 


+ (0) 


+ (0) 


implies 
implies 
implies 
implies 


violate nonnegativity constraints. 


—X3 


=0 
+ x4 = 
5 =o) 


x = (3,3, 0, —1, 0) 
x® = (2,0, —2, 0, 3) 


5.3. The Simplex Algorithm 


(c) Points corresponding to each basic solution of part (b) are indicated on the plot of 
part (a). Confirming property , basic feasible x, x, and x® correspond to 
extreme points of the feasible region, while basic infeasible x and x correspond 
to infeasible points where active constraints intersect. 


THE SIMPLEX ALGORITHM 


5. 


The simplex algorithm is a variant of improving search elegantly adapted to ex- 
ploit the special properties of linear programs in standard form. Like all improving 
searches, it moves from solution to solution, retaining feasibility and improving the 


objective function until a local optimum (global by [54)) is encountered. 

What is unusual is that every step of the simplex leaves us at an extreme-point 
solution. Under property this means that we only need to think about moves 
among basic feasible solutions. 


Standard Display 


Definition showed how linear programs in standard form can be compactly 
described by a cost vector c, a constraint coefficient matrix A, and a right-hand-side 
vector b. The evolution of a simplex search will produce many more vectors for 
current basic solutions, move directions, and so on. 

In this book we display all these vectors simply by adding them as rows of a 
growing table. To see the idea, return to our Top Brass Trophy model (5.2). Standard- 
form data produce an initial table, with columns for each of the variables and the 
right-hand side: 


xy x4 x3 X4 X5 X6 
max ¢ 12 9 0 0 0 0 b 
1 0 1 0 0 0 1000 
A 0 1 0 1 0 0 1500 
1 1 0 0 1 0 1750 
4 2, 0 0 0 1 4800 


Initial Basic Solution 


Any improving search begins by choosing a starting feasible solution, and simplex 
requires an extreme point. 


5.20 | Simplex search begins at an extreme point of the feasible region (i.e., at 
a basic feasible solution to the model in standard form). 


To illustrate for the Top Brass example, we choose (arbitrarily) to begin at 
extreme point x© = (0,0) in Figure 5.4. Table 5.1 will show that this solution is 
obtained as the basic feasible one for basic variables B = {x3, x4, x5, x6} and nonbasic 
columns N = {x1,x2}. Adding this basis and the corresponding solution vector to 
our table gives 
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XxX, XQ *3 x4 x5 x6 

Macs 12919, 0) 0 0 0 b 
{i 20 1 0 0 0 1000 

A (eceiail 0 1 0 0 1500 
jem al 0 0 1 0 1750 
i) 0 0 0 1 4800 
NIDN nee B B B 

x09) 0 0 1000 1500 1750 4800 


If we did not already know the solution from Table 5.1, it would have been computed 
by setting nonbasics x; = x2 = 0 and solving as in definition : 


Simplex Directions 


The next requirement is move directions. We want simplex to follow edge directions 
joining current extreme points to adjacent ones. Each such edge direction follows a 
line determined by all but one of the active constraints that fix our current extreme 
point because definition | 5.14] requires that adjacent extreme points share all but 
one of their determining active constraints. But the active constraints at a basis 
feasible solution are nothing more than the nonnegativity constraints on nonbasic 
variables. 

We obtain the simplex directions by making the nonbasic nonnegativities inac- 
tive one at a time. 


That is, Ax; = 1 on the increasing nonbasic, Ax; = 0 on other nonbasics, and basic 
components are obtained by solving conditions for a feasible direction in the equality 
constraints. 

There is one simplex direction for each nonbasic variable. For example, at cur- 
rent solution x of the Top Brass example, we have one simplex direction increasing 
nonbasic x;, and another increasing xp. 

Simplex directions always have +1 on the increasing nonbasic, and other non- 
basic components = 0. To complete the directions, we must determine components 
for the basic variables. 

Reaching all the way back to principle | 3.25 | in Section 3.3, we can see what is 
required. A direction Ax follows the equality constraint 


ye ajXj =b 
j 
if and only if it satisfies the net-change-zero condition 


gt Ax; = 0 
j 


Thus for our entire system of equality constraints 
Ax=b 
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every feasible direction must satisfy 
AAx=0 (5.10) 


At current Top Brass solution x, condition (5.10) produces the following 
equation system for the simplex direction increasing x; when Ax; is fixed = 1: 


1d) OC) +1Ax3 +0Axy, +0Ax5 +0Ax5 = 0 
+0(1) +10) +0Ax3 +1Axqg +0Ax5 +0Ax6 = 0 
+171) +10) +0Ax, +0Ax, +1Axs +0Axe = 0 
+4(1) +20) +0Ax3 +0Axq +0Axs +1Ax6 = 0 

Corresponding equations for the simplex direction increasing x2 are 

+100) +00) +1Ax3 +0Ax, +0Axs +0Ax6 = 0 
+0(0) +10) +0Ax3 +1Ax, +0Axs +0Axe = 0 
+100) +10) +0Ax3 +0Axq +1Axs +0Ax6 = 0 
+4(0) +2(1) +0Ax3 +0Axg +0Ax5 +1Axe = 0 


Will these systems have a solution? Absolutely. A basis isa collection of column 
vectors that can represent every other vector of the same dimension, and we have 
been careful to leave undetermined only the components on basic variables. Each of 
the systems (5.10) amounts to finding multipliers Ax; that weight columns of basic 
variables to produce the negative of the column for nonbasic k. Such multipliers 
have to exist. In fact, we know that they are unique (Primer 4). 

Completing the two simplex directions for our current Top Brass solution yields 
the following updated table: 


xy X2 x3 X4 X5 X6 
max ¢ 1 9 0 (0) 0 0 b 
1 0 1 0 0 0 1000 
A 0 1 0 1 0 0 1500 
1 1 0 (0) 1 0 1750 
4 2 0 0 0 1 4800 
N WN B B B B 
x(0) 0 0 1000 1500 1750 4800 
Axforx, 1 0 -1 0 -1 4 
Axforx, 0 1 0 -l -1 =o 


SAMPLE EXERCISE 5.11: CONSTRUCTING 


ee 


A minimizing, standard-form linear program has the following coefficient data: 


xy x2 x3 X4 
mine 2 0 —3 18 b 
A 1 -1 Zz 1 4 
1 1 0 3 ep; 


Assume that x; and x3 are basic, solve for the current basic feasible solution, and 
compute all corresponding simplex directions. 
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Analysis: Following definition 5.16], the current basic solution is obtained by set- 
ting nonbasics x. = x4 = 0 and solving equality constraints for basic variable values. 
Here, 

+14, —10) +2%3 +1(0).=4 

tix, +1(0) +0x3 + 3(0) = 2 
gives x = (2,0, 1, 0). 

There will be two simplex directions, one increasing nonbasic x2 and another 
increasing nonbasic x4. Following construction 5.21 |, the direction increasing x2 has 
Ax2 = 1 and its components on all other nonbasics (here only x4) = 0. We must solve 
for basic components Ax; and Ax; satisfying net-change-zero condition (5.10). The 
corresponding linear system 

4+1Ax, —10) +2Ax3 +1(0) = 0 
41 Ax, +10) +0Ax3 +3(0) = 0 
has unique solution Ax; = —1, Ax3 = 1. Thus the simplex direction increasing x2 is 
Ax = (—1, 1, 1,0). 
For Ax the corresponding linear system is 
+1Ax, —10) +2Ax3 +1(1) =0 
+1 Ax, +100) +0Ax3 +3(1) =0 
Solving for the two unknown components yields simplex direction Ax = (—3, 0, 1, 1). 


Improving Simplex Directions and Reduced Costs 


Having constructed the collection of simplex directions that can be pursued from our 
current basic solution without losing feasibility, our next task is to see if any of them 
improve the objective function 


n 
fey)Ae-xA a CjXj 
j=l 
We know from Section 3.3 that improvement can be checked by referring to the 
gradient Vf(x) at our current solution. 
For linear objective function f(x), the gradient is just the vector of objective 
function coefficients. That is, 


Vig) Ses (Glo iG) for all x 


Thus gradient conditions and require checking quantities that we term 
reduced costs. 


improving for a maximize 
; > 0, and for a minimize 


5.3. The Simplex Algorithm 


These simple tests tell us immediately that both simplex directions at solution 
x of the Top Brass example improve the maximize objective. For example, the 
direction increasing x; gives 


c1 = (12, 9,0, 0, 0,0) - 1,0, -1, 0,0, -1, -4) =12>0 
and for the one increasing x», 
C2 = (12, 9,0, 0,0, 0) - (0,1,0, =1, -1, -2) =9>0 


More typically (see Sample Exercise 5.12), some simplex directions at a current basic 
solution will improve, and others will not. 


Analysis: We apply computations and . For direction Ax = (—1, 1, 1, 0) 
increasing x, 


i= 2,0, 3.18) at it = —5 20 


Thus the direction does improve. On the other hand, for Ax = (—3, 0, 1, 1), increas- 
ing x4 does not improve because 


C4 = (2,0, —3, 18) - (—3,0,1,1) =9 40 


Step Size and the Minimum Ratio Rule 


Simplex can adopt for the next move of improving search any simplex direction that 
improves the objective function. The next issue is “How far?”, that is, what step size 
A should be applied to chosen direction Ax? 

Following principle in Section 3.2, we would like to take the biggest step 
that preserves feasibility and improves the objective function value. With the con- 
stant gradient for linear programs, an improving simplex direction remains improving 
forever. Also, we constructed simplex directions to maintain all equality constraints 
Ax =b. 

If there is any limit on step size A, it must come from eventually violating a non- 
negativity constraint. That is, some component of the revised solution must become 
negative. With all components feasible, and thus nonnegative at our current solu- 
tion, infeasibility can occur only if some component of the chosen simplex direction 
is negative. The first solution component forced to zero fixes 2. 


= ratio computation —__ 
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To illustrate, we arbitrarily set Ax = (1,0, —1, 0,0, —1, —4), increasing x; at 
x of our Top Brass example. The chosen direction Ax" does have negative com- 
ponents, so there is no indication of unboundedness. 

To decide the maximum step, we add a row to our table that computes the step 
size at which each component would drop to =0. 


xy X2 x3 X4 Xs X6 
N N B B B B 
x) 0 0 1000 1500 1750 4800 
Axe libero =i 0 =i -4 
1000) _ 1750 4800 
-(-1) -~-1) -(-4) 


The least of these ratios establishes A in rule | 5.24 |: 


. {1000 1750 4800 
ya ——_ , —_ , —_ 7 = 
min tattoo BA | 1000 


Thus our new solution is 
xD — xO 4, Ax 
= (0, 0, 1000, 1500, 1750, 4800) + 1000(1, 0, —1, 0, —1, —4) 
= (1000, 0, 0, 1500, 750, 800) 


| SAMPLE I 


Let the current simple 
(13, 0, 10, 2, 0, 0), with x. and xs nonbasic. Determine the maximum step and new 
solution (if any), assuming that each of the following is the improving simplex direc- 
tion associated with increasing x2. 


(a) Ax = (12, 1, —5, —1, 0, 8) 
(b) Ax = (0, 1, 6, 35 0, t) 

(c) Ax = (—1, 1, —8, 0, 0, —5) 
Analysis: 


(a) This improving simplex direction has negative components, so we apply rule 


: 10 2 
A = min ; ——-, ——; =2 
eS) a) 
Notice that values for x; were not included in this computation; with a positive 
directional component Ax; = 12, x; is increasing. The new simplex solution will be 
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x78) ~ x77) 4) Ax 
= (13, 0, 10, 2, 0, 0) + 2(12, 1, —5S, —1, 0, 8) 
=1(875 2.1050) 02516) 


(bd) This improving simplex direction has no negative components, so progress is 
_ unlimited. Under principle [5225 , the model is unbounded. 


(c) This improving simplex direction does have negative components. Applying rule 


5.24 | yields 


1 
i= min | i a0 Y 0 


1)" =(@8)" +65) 
_ and new solution 
x9). xd? LO Ag = xcD 
The zero step 4 results from basic variable xg happening to take on the zero value 


more typical of nonbasics—a common occurrence with large-scale linear programs. 
In Section 5.7 we discuss this degenerate case further. 


Updating the Basis 

Step size rule tells us to continue from our present extreme-point solution, 
along the edge direction formed by increasing a single nonbasic, until an adjacent 
extreme point is formed by a newly active nonnegativity constraint (definitions 


15]). To continue the algorithm we need to find a new basis corresponding to 
this new extreme-point solution. Active nonnegativity constraints tell us how. 


26 _ After each move of si search, the nonbasic variable generating ‘the 
chosen simplex direction enters the basis, and any one of the ossibly several) | 
basic variables fixing step size a deaves the Basis” 


That is, we move to the new basis implied by the nonnegativity constraint on the 
increasing nonbasic variable becoming inactive, while a nonnegativity constraint on 
a blocking basic becomes active. 

For our Top Brass example, the move from x increased nonbasic x;, and x3 
is the first component to drop to =0. Thus x; enters and x3 leaves, resulting in new 
basic set {x1, x4, x5, x6}. 

Figure 5.5 interprets graphically. New solution x™ is indeed the adjacent ex- 
treme point obtained when we move along the edge direction for x; until the non- 
negativity constraint for x; becomes active. The new basis makes x; basic and x3 
nonbasic. 


SAMPLE XE! CISE 5.14: UP 


Return to Sample Exercise 5.13 and determine for boandell cases (a) and (c) what 
variables should enter and leave the basis. 
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* 


x, < 1000 (x, = 0) 


x, $1500 (x, 20) 


Xi 


FicurE 5.5 Simplex Search of the Top Brass Example 


Analysis: We apply rule . Increasing nonbasic x2 enters in both cases. For case 
(a) there is a choice of leaving variables because both x3 and x4 establish A; either 
one could be selected to leave the basis. Variable x¢ alone fixes the value of A in case 
(c); it must be chosen as the leaving basic. 


Rudimentary Simplex Algorithm 


We have now developed all the main ideas of simplex search. Simplex implements 
extreme-point scarch by moving from basic feasible solution to basic feasible solution 
until either the problem is shown to be unbounded or the current solution proves 
optimal. At every iteration a move is contemplated in each simplex direction. li 
any of the simplex directions is improving, one is selected and pursued as far as 
feasibility will permit. If no simplex direction improves, we stop and report an 
optimum. Algorithm 5A gives a formal statement. 


ALGORITHM SA: RUDIMENTARY SIMPLEX SEARCH FOR 


Step 0: Initialization. Choose any starting feasible basis, construct the — 
corresponding basic solution x, and set solution index t < 0. 
Step 1: Simplex Directions. Construct the simplex direction Ax associated 
with increasing each nonbasic variable x;, and compute the corresponding 
reduced cost ¢; = c- Ax. 


5.3. The Simplex Algorithm 


Step 2: Optimality. If no simplex direction is improving (no G > 0 for a 
maximize problem, or no ¢; < 0 for a minimize), then stop; current solution 
x is globally optimal. Otherwise, choose any improving simplex direction 
as Ax“*) and denote the associated entering basic variable x». 

Step 3: Step Size. If there is no limit on feasible moves in simplex direction 
Ax‘*? (all components are nonnegative), then stop; the given model is 
unbounded. Otherwise, choose leaving variable x, so that 

©) au) ) 


xr : i (t+1) 
= min { —/___: Ax! Oi ign Seka ee eee 
— Axi) J oe Ax ft) 


Step 4: New Point and Basis. Compute the new solution 


xf) - yO 4) Age 


and replace x, in the basis by x,. Then advance t — ¢+ 1, and return to 
Step 1. 


Rudimentary Simplex Solution of Top Brass Example 


Table 5.2 details the full Algorithm SA search of our Top Brass Trophy example in LP 
standard form. Figure 5.5 tracks progress graphically. As in our earlier discussion, 


TABLE 5.2 Rudimentary Simplex Search of Top Brass Trophy Example 


X, % q x, x X% 
max ¢ 12 9 0 0 0 b 
1 0 1 0 0 0 1000 
A 0 1 0 1 0 0 1500 
1 1 0 0 1 0 1750 
4 2 0 0 0 1 4800 
t=0 N N B B B B 
x 0 ) 1000 1500 1750 4800 c-x9=0 
Ax for x; 1 0 -1 0 -1 -4 q =(12 
Ax for x» 0 1 0 -1 =1 -2 @=9 
a a 1000 1750 4800 A= 1000 
-CD SE) -(-4) 
t=1 B N N B B B 
x) 1000 0 0 1500 750 800 c- x) = 12,000 
Ax for x5 0 1 0 =i =i 2 @= 
Ax for 4 zi 0 1 0 1 4 @=-12 
= ae 1500: 130 _800_ A= 400 
-(-1) -(C1) =(€2) 
t=2 B B N B B N 
x) 1000 400 0 1100 350 0 c- x = 15,600 
Ax for -l 2 1 -2 -1 0 Gz 
ON ai Wee en cremated Baas ft tie el ene oe ne Ce 
1000 om. _& 1100 350_ er | A= 350 
-(¢1) -(-2) -(1) 
t=3 B B B B N N 
x) 650 1100 350 400 0 0 c-x®) = 17,700 
Ax for x, 1 -2 -1 2 1 0 6 =-6 
Ax for x5 0.5 0.5 0.5 -0.5 0 1 G=-1.5 
“optimal” 


ne 
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the search begins at basic feasible solution x = (0,0, 1000, 1500, 1750, 4800). We 
have already detailed the first iteration t = 0. Simplex directions for nonbasics x; 
and x2 both improve. Choosing p = 1, the maximum feasible step is determined by 
the component with subscript r = 3 at A = 1000. 

After the move, we have new solution x“ = (1000, 0, 0, 1500, 750, 800) with 
objective function value $12,000. Increasing variable x; has replaced blocking 13 in 
the basis. The process now repeats with ¢ = 1. Simplex directions are available for 


x2 and x3, but only the first improves because ¢; = 9 and ¢3 = —12. Thus p = 2, and 
we take a maximum step in the direction for x2. Step size 4 = 400 is fixed as basic x5 
decreases. 


Taking this new step brings us to x = (1000, 400, 0, 1100, 350, 0) with objec- 
tive value $15,600. Figure 5.5 confirms that it too is an extreme-point solution. 

The basis is now {x;,X2,xX4,Xs5}. As usual, there are two simplex directions 
available, one for each nonbasic. However, only the one for x3 improves. At 4 = 350. 
the nonnegativity constraint on x5 becomes active, producing new solution xo 
(650, 1100, 350, 400, 0, 0). 

With t = 3 we repeat the sequence again. This time, however, neither simplex 
direction is improving. The search terminates with (global) optimal solution x* = 
(650, 1100, 350, 400, 0, 0) having profit $17,700. 


Stopping and Global Optimality 


If Algorithm 5A stops with a simplex direction along which we can improve forever 
without losing feasibility, the given model is clearly unbounded. But what if it stops 
with an optimum? 

Simplex search considers only the simplex or edge directions as it looks for 
an improving feasible move. As long as one of the simplex directions improves. 
this approach is completely consistent with our previous development of improving 
search. Still, we stop when no simplex direction is improving and feasible. 

Could some non-simplex feasible direction still improve? No. 


To see why, notice that any feasible direction at a current basic solution must 
increase nonbasics because the basic part of the solution is determined uniquely 
once nonbasics are fixed at =0. The key insight is that any non-simplex direction 
(i.e., any that increases more than one nonbasic) will just be a weighted sum of 
simplex directions. For example, at the last iteration of the Top Brass search, we 
might have considered a direction Ax increasing both xs and x5. Say that Axs = 3 
and Ax¢ = 7. The only way this choice of nonbasic components can retain net change 
zero condition (5.10) is for Ax to have the form 


Ax = 3(Ax) + 7(Ax’) 


where Ax and Ax’ are the simplex directions for x5 and x6, respectively. It follows 


5.4 Dictionary and Tableau Representationsof Simplex 


that 
c-Ax = ec: (3Ax +7Ax’) 
= 3(c-Ax) +7 (c-Ax’) 
= 3(—6) + 7(—L5) 
= —29.5 
We sce that no positive combination of simplex directions can meet the test for 


improvement unless at least one simplex direction does, too. A current basic feasible 
solution is optimal if and only if no simplex direction is improving. 


& | DICTIONARY AND TABLEAU REPRESENTATIONS 
OF SIMPLEX 


Our development of the simplex algorithm in Section 5.3 follows the improving search 
paradigm, which constitutes one of the main themes of this book. It also reflects the 
way professionals and researchers in linear programming think about simplex. 

Still, readers who have encountered simplex in other introductory texts may 
find our form a bit difficult to recognize. In this section we make a brief connection 
to more traditional formats. Those not confused by our departures from tradition 
can skip the section without loss. 


Simplex Dictionaries 


Traditional introductory developments of the simplex algorithm view the process 
that we have conceived as a search in terms of manipulating the objective function 
and main standard-form equations 


n 
YS aux; = bi for alli=1,...,m 
j=l 


At each iteration representations called simplex dictionaries express the basic vari- 
ables and the objective function value in terms of the nonbasics. 


aid basic vari- 


Dictionary form is achieved by Gaussian elimination—solving for one basic 
variable at a time and substituting for it in other constraints and the objective. To 
see the idea, consider the basis corresponding to t = 2 in Table 5.2. There basic and 
nonbasic index sets are 
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We begin our derivation of the corresponding dictionary with the original ob- 
jective and constraints: 


z= 12x, + 9x2 
+3 = 1000 
xX) +x4 = 1500 


Xy +x. +x5 = 1750 
4x, + 2x2 + x6 = 4800 


The first constraint expresses basic variable x; as 
x; = 1000 — (1x3) 
Substituting for x; leaves 


x1 = 1000 — (1x3) 

Xg2+x%4 = 1500 

(1000 — x3) + x2 +2%5 = 1750 
4(1000 — x3) + 2x2 +X%6 = 4800 


We now continue with basic x2. Solving the second equation and substituting 
gives 


x; = 1000 — (1x3) 

x2 = 1500 — (1x4) 

(1000 — x3) + (1500 — x4) + x5 = 1750 
+4(1000 — x3) + 2(1500 — x4) + x6 = 4800 


The third equation can now be solved for basic variable x4. After substitution 
and solving the last equation for x5, we complete the constraints as 


x1 .=. 1000 — (+1x3 +0 Xe¢) 
Co — : 400 — (—2x3- +0.5x6) (5.11) 
Xa =. MOO (42x3 —0.5x6) 
xs = 350 — (41x3 —0.5x6) 
Finally, we substitute these expressions in the objective function to obtain 
z = 12(1000 — 1x3) + 9(400 — 2x3 — 0.5x6) (5.12) 


= 15,600 + 6x3 — 4.5x6 
The dictionary (5.11)-(5.12) is now complete. In the notation of definition 
5.28], for example, Z = 15,600, ¢; = 6, by = 400, and G46 = —0.5. 


The eon eae directions ane edicel costs for ie linea program of Sample 
Exercises 5.11 and 5.12 
min z= 2x; — 3x3 + 18x4 
st. xX, —X2 + 2x3 +x4 = 4 
xy +x. + 3x4 =) 
XN Mad = 0) 


LC iti iach 


5.4 Dictionary and Tableau Representationsof Simplex 
had x; and x3 basic. Apply Gaussian elimination to derive the corresponding simplex 
dictionary. 


Analysis: We want to solve for basic variables x; and x3 in terms of nonbasics. From 
the first constraint, 


x, =4— (—1x2 + 2x3 + 1x4) 
Substituting in the second constraint yields 
(4+ x2 — 2x3 — 1x4) +42 +3x4 =2 


Then, solving for x3 and substituting produces the dictionary: 


Zz = 1 + =5x + 9x4 
xp 2 = GE +t 3x4) 
x3~= i (-—1x2 = 1x4) 


Simplex Tableaux 


Simplex tableaux are detached-coefficient displays of exactly the same information 
as simplex dictionaries. 


For example, the dictionary (5.11) corresponds to the simplex tableau 


xy X2 x3 X4 X5 X6 

0 oOo -6 O 0 44.5 15,600 
1 0 ire) 0 0 1,000 
0 1 2 0 0 0.5 400 
0 0 2 1 0 -0.5 1,100 
Oro) Leh 0 1 -0.5 350 


The only change is that nonbasic variables have been translated to the left-hand side, 
and coefficients have been extracted in a matrix. 


SAMPLE EXERCISE 5.16: CONSTRU 


Construct the simplex tableau corresponding to the dictionary of Sample Exer- 
cise 5.15. 


Analysis: With all variables translated to the left-hand side, the coefficient tableau 
is ; 
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Simplex Algorithm with Dictionaries or Tableaux 


In dictionary/tableau form, simplex still moves from basic feasible solution to basic 
feasible solution. Each simplex iteration begins by checking the sign of objective 
function coefficients ¢; on nonbasic variables. If none is negative for a minimize 
problem (positive for a maximize), the current basic solution is optimal. Otherwise, 
the solution can be improved by increasing a nonbasic variable from its basic-solution 
value =0. We pick one such variable as the entering nonbasic. Its coefficient column 
in the dictionary or tableau tells us how basic variables will change as the nonbasic 
increases. If it can increase forever without losing feasibility, the problem is un- 
bounded. Otherwise, the entering variable’s increase eventually drives some basic 
variable to its lower limit of zero. This establishes a leaving basic variable. We update 
the dictionary or tableau to the new basis and repeat the process. 


Correspondence to the Improving Search Paradigm 


This brief synopsis of the simplex with dictionaries or tableaux should sound very 
familiar. It is, in fact, exactly Algorithm 5A. The only difference comes in how the 
arithmetic is accomplished. 

Think first about the current basic solution. When nonbasic variables are fixed 
=0, dictionary format makes the corresponding values of basic variables obvious. 


Ss 30 - Right-hand-side constants b;, of simple) : ictionary/tablee format show 
current values for corresponding basic variables xx. Similarly, Zis the ob ctive 
__ function value of the current basic solution. ae 


‘ 


For example, the dictionary (5.11) shows clearly that with nonbasics x3 = x6 = 

(0, the current basic solution is 

(b1, bz, 0, ba, bs, 0) = (1000, 400, 0, 1100, 350, 0). 
This is exactly the solution reported in Table 5.2 for t = 2. It objective function value 
is Z = 15,600. 

Next consider the simplex directions. Definition [5.21 | [5.21 | specifies Sommoncare for 
nonbasic variables and leaves those for basic variables to be computed. Specifically. 
the changes in basics associated with increasing any nonbasic are unique solutions 
to equations AAx = 0. 

The simplex in dictionary/tableau form uses columns of the tableau to predict 
the same basic variable changes. Since the changes are unique, there must be a close 
connection to simplex directions. 


basic variables A; are 
snis in the seals Ge 


‘Tections 1 increasing ae 


The dictionary (5.11) illustrates again. There 
43 =+1 a6 =0 
3 =—2 fi2,6 = +0.5 
43 = = 2 a46 = —0.5 
a53 = +1 as,6 = —0.5 


5.5 Two Phase Simplex 


Table 5.2 for t = 2 shows that the simplex directions are 


Ax, —! Ax, 0 

Ax2 +2 Ax2 —0.5 

Ax3 = +1 Ax3 $i, 0 

area Cot eae for x3; and ec ce: 05 for X6 
Axs —1 Axs 0.5 

AX6 0 AXx6 1 


Notice that tableau coefficients are exactly the negatives of corresponding simplex 
direction components for basic variables x1, x2, x4, and xs. 

Finally, focus on the reduced costs which tell us whether a nonbasic variable 
can increase productively. We have employed the same c; notation in both reduced 


cost definition and format because they refer to the same quantities. 


Be Simplex dictionary/tableau objective function coefficients G are exactly 
_ the reduced costs of definition | 5.22 |. _ 


In both representations they show how objective function value z will change per 
unit increase in nonbasic variable x; if basics are adjusted to preserve feasibility in 
main equality constraints. For example, the reduced cost of x3 in both dictionary 
(5.11) and Table 5.2 (at t = 2) is c3 = 6. 


Comparison of Formats 
Observations to document how solving for current basic variables in 


dictionary format is just an alternative way to compute the basic solution, 
simplex directions, and reduced costs required in rudimentary simplex Algorithm 
5A. Nothing is fundamentally different. 

One can argue about which computation is easier for hand calculation. Still, 
it is important to realize that neither is ever employed in serious computer imple- 
mentations of the simplex algorithm. Sections 5.7 and 5.8 will detail the much more 
efficient revised simplex version at the heart of all commercial simplex codes. 


§ % Two PHASE SIMPLEX 


As usual, we have developed the simplex version of improving search assuming that 
we know a starting basic feasible solution. In most real problems we will have to 
search for one. 

In Section 3.6 we introduced the two-phase approach we will adopt (Algorithm 
3B). A solution is chosen that satisfies at least part of the constraints; artificial vari- 
ables are added to synthetically satisfy all other constraints; Phase I minimizes the 
sum of the artificials; Phase II proceeds from the result of Phase I to optimize the 
real objective function. 


EXAMPLE 5.2: CLEVER CLYDE 


We will illustrate two-phase simplex computation with the contrived but instructive 
case of Clever Clyde. Clyde is an entrepreneur of dubious integrity who is seeking 


pax 


wig Chapter Five e Simplex Search for Linear Programming 


to open a sports collectibles business. He has only $5000 of his own, and the business 
will require at least $100,000. Still, Clyde also has a real knack for convincing others 
to invest in his projects. One investor has already agreed to pay 50% of the initial 
cost of the firm in return for an equal partnership, and Clyde has an appointment 
tomorrow with another prospect. We want to consider several cases for the fraction 
of support, call it w, that Clyde may get from selling a second “equal” partnership. 
His goal is to maximize the size of the business. 

To formulate Clyde’s dilemma as a linear program, we will use the decision 
variables 
amount invested by present investor 1 (thousands of dollars) 
amount invested by new investor 2 (thousands of dollars) 
amount Clyde invests of his own money (thousands of dollars) 


x4 
x2 
X3 


[> [> 


|> 


Then clearly the objective function is to 
max x1 +X. +.x3 (5.13) 
Some constraints are also easy: 


Xy +x2 + x3 > 100 


x > 0, x. >0,5>x3>0 (5.14) 


Total investment must be at least $100,000, all amounts are nonnegative, and Clyde’s 
funds are limited to $5000. 
Investment constraints are a bit more complex. If investor 1 is to provide half, 
the capital, and investor 2 a fraction a, we want 
a FL 2 «sO Surcahidnosesee ae ety (5.15) 
Xy +%X2+%3 Xy +X. £3 
These ratio constraints do not even look linear, but knowing x; + x2 + x3 will be 
positive in feasible solutions, we can clear denominators and collect terms (principle 


[4.5]) to obtain 


—0.5x; + 0.5x2. + 0.5x3 = 0 


ax14 + (a—-—1)x2+ax3 =0 alo) 


Adding slacks x4 and x5 to place the LP in standard form, Clyde’s dilemma is 
represented by the model 


max +1x; + 1x2 + 1x3 
s.t. +1x + 1x2 + 1x3 —1x4 = 100 
+ 1x3 +1x5 = 5 
—0.5x, +0.5x +0.5x3 = 0 Ce 
“Oleh: Sehr se sey strats = 0 


oiled, Gna) eavea inn oie) 


Starting Basis in the Two Phase Simplex 


Two-phase computation begins by constructing a starting feasible solution to model 
(5.17) using artificial variables. There is one new issue in adapting the two-phase 
approach to simplex. Simplex search employs basic solutions. Our starting solution 
in Phase I must not just be feasible in the artificial model, but basic feasible. 


eae _— 


5.5 Two Phase Simplex 


One set of columns that is sure to qualify for a basis by being linearly inde- 
pendent is a collection where each column has only one nonzero component and no 
two of the columns are nonzero in the same component. That is, constraint columns 
with a single nonzero entry provide natural choices for a starting basis. Where none 
exists after the model has been placed in standard form, the column of an artificial 
variable will do the job. Either way, the single nonzero must have the same sign as 
the corresponding right-hand-side coefficient, so that the basis will produce a feasible 
basic solution. 


5A starting basis for ‘simplex can be obtained by making basic one variable 
for each constraint ae with that variable having its only. nonzero coefficient 


We can illustrate with Clever Clyde standard form (5.17). Our task is to find (or 
manufacture with artificial variables) a starting feasible basis. In the first constraint 
row we have a variable with its only nonzero coefficient there—slack x4. However, 
the coefficient of x4 is negative, while the right-hand-side 100 is positive. An artificial 
variable x6 will be necessary. 

Things go somewhat easier in the second row. There xs has its only nonzero 
coefficient with a sign that matches the right-hand side. Variable x5 will be part of 
the initial basis. The last two constraints of standard form have no variable with its 
only nonzero coefficient in their rows. Artificials x7 and xg will have to be added. 

Summarizing, simplex will begin Phase I with the data 


xy Xx X3 x4 X5 Ke x7 Xs 
max ec 1 1 1 0 0 0 O50 
mind 0 0 0 0 0 1 il 1 b 

1 1 1 =] 0 1 Os Oe ert00 
A 0 0 1 0 1 0 OF) 5 
-0.5 0:5= 00:5) 620 0 0 1 0 0 
(of a-1 a 0 0 0 0 il 0 

N N N NB B (BaD. 

x(0) 0 0 0 OSs loot 0134 0 


Initial solution x is the basic (artificially) feasible one obtained by assigning the 
value zero to all nonbasics and solving for the basics. 

Notice that we now have two objective function rows. The vector d denotes 
the objective function vector for Phase I (sum of artificials). We first minimize d- x, 
then return to maximize ¢- x in Phase II. 


SAMPLE EXERCISE 5.17: CONSTRUCTI ING > AN 


Introduce artificial variables as necessary to construct a Phase I artificial model and 
corresponding starting (artificially) feasible basis for the following standard-form 
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linear program: 


min 14x —9x3 +X4 
s.t. 8x, +X -— x3 yl 
4x> 7x3 +xq =—22 
es a ee = 


Xi, %25,X3, X4-—0 


Analysis: We apply principle [5.33 |. Variable x; appears only in the first main con- 
straint and has the same sign as the right-hand-side 74. Thus it can be the basic 
variable for that constraint. Variable x4 occurs only in the second main constraint, 
but its sign differs from the right-hand side; artificial variable xs will be needed. An- 
other artificial x6 is needed in the last constraint because no variable has its only 
nonzero coefficient there. Summarizing, the artificial model will be 


min x5 +X%6 


st. 8x, + xX. -— %3 = TA 
+4x%. —7x3 +X4 —X5 = —22 
ate Coa eX +x = 11 


X1»X2,%3, %45.X5,.46, 2.0 


The starting (artificially) feasible basis is x1, x5, and x6. 


Three Possible Outcomes for Linear Programs 
Recall that there are three possible outcomes for a linear programming model: 


_5.34 | A linear program may be infeasible (have no feasible solutions), be un- 
bounded (have arbitrarily good feasible solutions), or have finite optimal solu- 
tions. a ee 


In the discussion of Section 3.6 we detailed how we detect each case, and Algorithm 
5B provides details for the simplex. The problem is infeasible if Phase I reaches op- 


timality without reducing the sum of artificials to =0 (principle ). Otherwise it 
has feasible solutions (| 3.38 | ). Phase II then either detects unboundedness ( ) 


or stops on an optimal solution ( . 
Clever Clyde Infeasible Case 


The Clever Clyde example of this section was contrived so that we can illustrate all 
three possibilities for LP outcomes and know by easy deduction that simplex is draw- 
ing the right conclusion. Think for a moment about what Clyde needs from investor 
2. His own $5000 would provide 5% of the $100,000 minimum for the business, and 
investor 1 has guaranteed 50%. If w < 0.45, there can be no feasible solution. 

Take a = 0.4. Table 5.3 shows the Phase I simplex computations leading to a 
conclusion of infeasibility. Beginning with xs, x5, x7, and xg basic, the sum of artificial 
variables is d- x = 100. There are 4 available simplex directions, and 3 improve the 
objective. (Remember that we are minimizing, so that dj < QO implies improvement.) 
The direction increasing nonbasic x3 is chosen. 
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Step 0: Artificial Model. If there are convenient starting feasible bases for 
the given standard-form linear program, identify one, and proceed to Step 
3. Otherwise, construct an artificial model and artificially feasible basis by 


adding (or subtracting) artificial variables as in principle FS 33 


Step 1: Phase I. Beginning from the artificially feasible basis of Step O, 
apply simplex search to minimize the sum of the artificial variables subject 


to the constraints of the artificial model. 

Step 2: Infeasibility. If Phase | search terminated with a minimum having 
artificial sum > 0, stop; the original model is infeasible. Otherwise, use the 
final Phase | basis in the artificial model to identify a starting feasible basis 
for the original model. 

Step 3: Phase Il. Beginning from the identified starting feasible basis, apply 
simplex search to compute an optimal solution to the original standard-form 
model or demonstrate that it is unbounded. 


The next step is to determine how far we can pursue the Ax for x3 without 
losing feasibility. Here, something new occurs. Basic variables x7 and Xg, which were 
already =0 in solution x, are both decreased by Ax. As a consequence, the largest 
possible step size is A = 0. 

We will have more to say about this degenerate case in the next section. For the 
moment we merely act as if A were small but positive. Variable x3 enters the basis, 
and x7 (one of the two variables establishing 4) leaves, 

The new basic solution x“) is identical to x, but the basis that determines it 
has changed. As a consequence, we have new simplex directions to consider. This 
time only the Ax for x2 improves. It, too, leads to a degenerate step of 2 = 0, but x» 
enters the basis and xg leaves. 

In iteration ¢ = 2 we finally see real progress. Nonbasic x, enters, and xs leaves 
after a step of A = 25. The objective function value (sum of artificials) is halved to 
d-x = 50, Iteration ¢ = 3 is similar, reducing infeasibility tod-x = 5. 

Here the progress stops. As we consider the four available simplex directions 
at iteration t = 4, we see all fail the test for improvement. That is, x is optimal in the 
Phase I problem. But the sum of artificials is still 5. No solution using only the original 
variables satisfies all constraints [i.e., the original model (with « = 0.4) is infeasible]. 


IMPLEX PHASE I 


Use two-phase simplex Algorithm 5B to establish that the linear program 


max 8x,+ 11x 
s.t. Xi + X92 <2 


X1 + X2 =3 MER] LA 
=anG ie PP 
X41, x2 = 0 LAPIN Taare! ° INSTITUUTTI 


has no solution. eho. 
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TABLE5.3 Simplex Computation for Clever Clyde Infeasible Case 


Bal x2 +3 x4 x5 6 x7 x8 
maxec 1 1 it 0 0 0 0 0 
mind 0 0 0 0 0 1 tf 1 b 
1 1 ai Ail 0 1 0 0 100 
A 0 0 1 0 1 0 0 0 5 
-0.5 0.5 0.5 0 0 0 1 0 0 
0.4 ~0.6 0.4 0 0 0 0 1 0 
t=0 N N N N B B B B Phase I 
x0) 0 0 0 0 5 100 0 0 d- x = 100 
Ax for x; 1 0 0 0 0 =i 0.5 0.4 d, =-0.9 
Ax for x5 0 1 0 0 0 =f ~0.5 0.6 d,=-0.9 
Ax for x3 0 0 1 0 = =f, -0.5 0.4 a; =E19 
Ax for x4 0 0 0 il 0 1 0 0 d= 1.0 
5 100 0 0 
1 1 0.5 0.4 A= 0.0 
eeetil N N B N B B B N Phase I 
x) 0 0 0 0 5 100 0 0 d- x) = 100 
Ax for x; 1 0 -1 0 1 0 1 0 d,=1.0 
Ax for x, 0 1 1.5 0 =5) 2915 =1.25 0 dy = 
Ax for x4 0 0 0 il 0 1 0 0 dy = 1.0 
Ax for xg 0 0 2015 0 DS 2.5 1.25 1 dg = 4.75 
a2 100 SOME E 
1.5 2.5 Ara 
t=) N N B B N Phase I 
x(2) 0 0 0 0 5 100 0 0 d- x2) = 100 
Ax for x; 1 0.8 0.2 0 -0.2 %) 0 0 ae 
Ax for x4 0 0 | 0 1 0 0 d,= 1.0 
Ax for x7 0 0.8 -12 0 2, DP eateiastl 0 d= 3.0 
Ax for xg 0 1 -1 0 1 0 0 1 dg = 1.0 
a apg i = 4 205 
2 
pos B B N N B N N Phase I 
x3) 25 20 5 0 0 50 0 0 d-x®) = 50 
Ax for x4 0 0 0 1 0 1 0 0 d4= 1.0 
Ax for x5 -5 -4 -1 0 1 10 0 0 ds = 10.0 
Ax for x7 6 4 0 0 0 -10 1 0 d, =9.0] 
Ax for xg 5) 5 0 0 0 -10 0 1 dg=-9.0 
= — == = = 0 se ile mS 
10 4H 
1=4 B B B N N N B N Phase I 
x) 55 40 5 0 0 0 5 0 dex =5 
Ax for x4 0.6 0.4 0 il 0 0 0.1 0 d= 0.1 
Ax for x5 1 0 -1 0 1 0 1 0 ds =1.0 
Ax for x¢ = (Ge wp 04: 0 0 0 1 -0.1 0 dg=0.9 
Ax for x¢ = 1 0 0 0 0 =I 1 dg = 0.0 
“infeasible” 


Analysis: We begin by including slack variables to obtain standard-form model 
max 8x; + 11x 
St. xX +X EAS =e 
xy +X2 —xX%, =3 
X1,X2,X3,X4 > 0 


5.5 Two Phase Simplex 


Variable x3 provides a starting basic variable in the first constraint, but artificial xs 
is needed in the second constraint. We are now ready to begin Phase I simplex with 
initial basis {x3, xs}. 


xy Xy X3 X4 Xs 
max ¢ 8 11 0 0 0 
mind 0 0 0 0 il b 
1 1 1 0 0 2 
A 1 il 0 -1 1 3 
t=0 es a N B Phase | 
x) 0 0 2 0 3 d-x=3 
Ax for x, 1 0 -1 0 -1 d, =[1 
Ax for x2 0 1 -1 0 -1 d,=-1 
Ax for x4 0 0 0 1 1 dy= +1 
S=Senn 
(teal B N N N B Phase I 
x) 2 0 0 0 1 d-x=1 
Axforx,  -1 1 0 0 0 dy =0 
Ax for x3 -1 0 1 0 1 dz =+1 
Ax for x4 0 On 1 1 d,=+1 
“infeasible” 


The original model is infeasible because the Phase I optimum has artificial total 1. 


Clever Clyde Optimal Case 


Suppose now that investor 2 is willing to provide 49% of the startup capital Clever 
Clyde requires. There should be a finite optimal solution. Table 5.4 traces Phase I 
simplex computation on this a = 0.49 case. The first two iterations closely parallel 
the infeasible variation of Table 5.3; two degenerate basis changes produce no actual 
progress. 

Iteration tf = 2 shows something new. The direction for entering nonbasic x; 
improves the objective, and the maximum feasible step of 4 = 50 erases all infeasi- 
bility. The resulting solution x is 0 in all artificial components and thus corresponds 
to a feasible solution in the original model. Iteration t = 3 confirms that it is optimal 
in the Phase I problem, with objective function value =0. 

With a known basic feasible solution, we are ready to pass to Phase II. Table 
5.5 details the computations. Now that only the standard-form variables are present, 
there is only one nonbasic. Its simplex direction Ax tests as improving for our 
maximize objective function, so it will enter the basis. 

A check of ratios shows that feasible progress will stop at A = 400 as x5 leaves 
the basis. The new solution is x” = (250,245, 5, 400, 0). Now the only simplex 
direction is that of x5 and it does not improve the objection function. We conclude 
that x" is optimal. If investor 2 will provide 49% of the capital, a $500,000 business 
is achievable, taking $250,000 from investor 1, $245,000 from investor 2, and the 
remaining $5000 from Clyde. 


DAT, 
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TABLE 5.4 Phase I Simplex Computation for Clever Clyde Optimal Case 


xy x2 +3 x4 x5 *6 x7 xg 
mind 0 0 0 0 0 1 1 it b 
1 1 1 =f 0 1 0 0 100 
A 0 1 0 0 1 0 0 0 5 
0.50 0.50 0.50 0 0 0 1 0 0 
0.49 -0.51 0.49 0 0 0 0 1 0 
t=0 N N N N B B B B Phase I 
x00) 0 0 0 0 5 100 0 0 dex) = 10 
Ax for x 1 0 0 0 0 zi 0.50 0.49 d, =-0.% 
Ax for x, 0 1 0 0 0 -1 0.50 0.51 d> = -0.%% 
Ax for x3 0 0 1 0 1 = -0.50 -0.49 d; =-198 
Ax for x4 0 0 0 1 0 1 0 0 d= 1.0 
5 100 0 0 an 
1 1 0.50 (0.49 ve 
t=1 N N B N B B B N Phase I 
x) 0 0 0 0 5 100 0 0 dex = 19 
Ax for x; 1 0 -1 0 1 0 it 0 d; =1.0 
Ax for x5 0 1 1.04 0 -1.04- =2.04 -1.02 0 d,-Bs 
Ax for x, 0 0 0 1 0 1 0 0 d,= 1 
Ax for xg 0 0 2.04 0 2.04 2.04 1.02 1 dg = 4.06 
= rad Jet = eg LOU) = 
oa 304) ep a 
ji N B N B B N N Phase | 
x) 0 0 0 0 5 100 0 0 d-x@) = 19 
Ax for x; 1 0.98 0.02 0 0.02 =) 0 0 d,=2S 
Ax for x4 0 0 0 1 0 1 0 0 dy= 1 
Ax for x, 0 0.98 1.02 0 1.02 2 1 0 dy = 3 
Ax for xg 0 1 -1 0 1 0 0 1 dg = 1k 
= = = = ES 100 b. ah =5( 
0.02 2 ame 
t=3 B B B N B N N N Phase I 
x) 50 49 1 0 4 0 0 0 d-x@) = 
Ax for x4 0.50 0.49 0.01 1 -0.01 0 0 0 d,=0 
Ax for x¢ 0.50 0.49 0.01 0 0.01 1 0 0 de=1 
Ax for x7 1 0 -1 0 1 0 1 0 d,=1 
Ax for x¢ 0 1 -1 0 1 0 0 1 dg=1 
“feasible 
SAMPLE EXERCISE 5.19: MOVING TO SIMPLEX PHASE II 
A standard-form linear program has variables x1, ..., x5. To begin two-phase simplex 
Algorithm 5B, we introduce artificial variables x¢,...,xg. Explain for each of the 


following Phase I outcomes how the algorithm should proceed. 

(a) Phase I optimum is x = (0, 0, 0, 0, 0, 0, 3, 6) with x;, x7, and xg basic. 
(b) Phase I optimum is x = (0, 2, 0, 0, 9, 0, 0, 0) with x1, x2, and xs basic. 
Analysis: 


(a) Here the minimum artificial total is (0 + 3+ 6) = 9. The model is infeasible 
(principle [3.39 ), So we terminate. 
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TABLE5.5 Phase II Simplex Computation for Clever Clyde Optimal Case 


xy X2 X3 X4 X5 
maxc 1 1 1 0 0 b 
1 it 1 =f 0 100 
A 0 1 0 0 1 5 
-0.50 0.50 0.50 0 0 0 
0.49 -0.51 0.49 0 0 0 
t=0 B B B N B Phase II 
xO) 50 49 1 0 4 e+ x) = 100 
Ax for x4 0.50 0.49 0.01 1 -0.01 t =[1.0] 
rey 3 
E: B os Pot 0.01 say 
=1 B B B B Phase II 
xD 250 245 5 400 0 e+ x = 500 
Ax for x5 —50 —49 =] -100 1 Z; = —100 
“optimal” 


(b) With a minimum artificial total of =0, the model is feasible (principle ). 
We proceed to Phase II simplex, beginning with feasible basis {x1, x2, x5}. 


Clever Clyde Unbounded Case 


If Clyde is particularly convincing, he might talk investor 2 into covering a full 50% of 
the investment. Clearly, the size of the business is then unbounded because Clyde’s 
partners will be financing the entire cost. 

Phase I simplex computation for the a = 0.50 case is almost identical to that 
of Table 5.4. Only a few numbers differ. Table 5.6 shows Phase II. The feasible 
basis provided by Phase I consists of x1, x2, x3, and xs, with basic solution x = 
(50, 50, 0, 0, 5). The only nonbasic, x4, yields an improving simplex direction. 


TABLE 5.6 Phase II Simplex Computation for Clever Clyde Unbounded Case 


x4 x2 X3 X4 X5 
maxc 1 1 1 0 0 b 
il i iL =! 0 100 
A 0 1 0 0 1 5 
—0.50 0.50 0.50 0 0 0 
0.50 0.50 0.50 0 0 0 
t=-0 B B B N B Phase II 
x(0) 50 50 0 0 5 e+ x = 100 
Ax for x4 0.50 0.50 0.1 1 0.0 ¢, =(L.0 
es == d=00 
“unbounded” 


Notice that this improving simplex direction has no negative components (com- 
pare with Table 5.5). That is, the direction is not moving toward any nonnegativity 
constraint. It follows that there is no limit on how far we may follow the direc- 
tion while retaining feasibility (principle [5.25|). Since every step in the direction 
improves the objective, the model must be unbounded. 
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5.6 DEGENERACY AND ZERO-LENGTH SIMPLEX STEPS 


Our development of the simplex algorithm as a form of improving search has im- 
plicitly assumed that each iteration makes positive progress toward an optimal so- 
lution by advancing from a current extreme point solution to a superior one. Ifa 
better extreme point is encountered at each move, it is not hard to see that sim- 
plex must eventually produce an optimal solution or show that none exists (see Sec- 
tion 5.7). 

Unfortunately, this is not always the case. It is much more typical for a simplex 
search to follow the pattern of Figure 5.6. Progress at some iterations is interspersed 
with periods of no advance. In this section we explore the degeneracy phenomenon 
in linear programming, which brings about simplex steps with no gain. 
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FIGURE 5.6 Typical Objective Function Progress of Simplex 
Search 


Degenerate Solutions 


Degeneracy happens in linear programming whenever more constraints are active 
than the minimum number needed to define a point. 

-m linear program is degener- _ 
ic variables are active (i.e. if some 


Se 


Basic solutions require nonnegativity constraints to be active for the nonbasics 
(definition ), but when nonnegativities also happen to be active for some basics, 
there are alternative choices for the basic set. 


_5.36 In the presence 
_ solution, © — 
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Figure 5.7 illustrates graphically. Any three of the five inequalities B, C, H, 
I, and G define extreme point x. In standard form, each of those constraints 
will correspond to a nonnegativity constraint on a slack variable. This means that 
(5 — 3) = 2 basic variables will have value =0. The solution is degenerate. 


o) 


Determine for each of the following basic variable sets and basic solutions to a linear 
program in standard form whether the solution indicated is degenerate. 


(a) B = {x1, x3, x4} for x = (3, 0, 7, 2, 0, 0) 
(b) B = {x1, x2, x5} for x = (0, 8, 0, 0, 1, 0) 


Analysis: We apply definition | 3.35 |. 


(a) This basic solution is nondegenerate because none of the nonnegativity con- 
straints on basic variables are active. 


(b) This basic solution is degenerate because basic variable x; = 0. 


Zero-Length Simplex Steps 
We can see how degeneracy inhibits simplex algorithm progress by returning to the 
Clever Clyde example of Table 5.3. Table 5.7 recapitulates the first three iterations. 
Notice that the first three basic solutions encountered were the same: 
x — x — x® — 0,0, 0, 5, 100, 0, 0) 

But this basic solution was computed by three different basic sets: 

esos ean Galore — 0) 

ite Xo gx fale ol 

{xa XA Seo mal 6 9) 
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TABLE 5.7 Degenerate Iterations in Clever Clyde Example 


xy Xp x3 X4 X5 X6 x7 Xg 
=) N N N N B B B B Phase I (min) 
x(0) 0 0 0 0 5 100 0 0 d- x = 100 
Ax for x; 1 0 0 0 0) -1 0.5 -0.4 d,=-0.9 
Ax for x7 0 1 0 0 0 -1 -0.5 0.6 d,=-0.9 
Ax for x3 0 0 1 0 1 1 0.5 -0.4 d3=E19 
Ax for x4 0 0 0 1 0 1 0 0 dy=1.0 
5 100 0 
H i 05 As 00 
t=1 N N B N B B B N Phase I (min) 
xl) 0 0 0 0 5 100 0 0 d-x() = 100 
Ax for x} 1 0 -1 0 1 0 1 0 d,=1.0 
Ax for x) 0 1 1.5 0 1.5 2.5 1.25 0 d,=23.75 
Ax for x4 0 0 1 1 0 1 0 0 d4= 1.0 
Ax for xg 0 0 2.5 0 25) 25 1.25 1 dy = 4.75 
5 100 = A =0.0 
1.5 2.5 1.25 
i= N B B N B B N N Phase I (min) 
x?) 0 0 0 0 5 100 0 0 d- x) = 100 
Ax for x; 1 0.8 0.2 0 -0.2 ~2 0 0 d, =E2.0) 
Ax for x4 0 0 0 1 0 1 0 0 dy= 1.0 
Ax for x7 0 -0.8 -1.2 0 1.2 2, 1 0 dz = 3.0 
Ax for xg 0 1 -1 0 1 0 0 1 dg= 1.0 
5 100 E 
— = = = a a a = A =25 
f=3 B B B N N B N N Phase I (min) 
x) 25 20 5 0 0 50 0 0 d- x) = 50 


The two positive components were basic in every case, but there are several choices 
for the two remaining basics at value =0. 

Simplex directions are structured to keep the equality constraints satisfied, and 
also nonnegativity constraints on nonbasic variables. But there is no guarantee that 
a basic component of the solution will not be decreased. If that component already 
happens to = 0, as it does in degenerate cases, the result may be a step of A = 0. 


a ° sollition may SEdune moves with steps A A= _ 0. 


This is precisely what happens on the initial two moves of Table 5.7. In the first 
case it was decreasing Axg = —0.4, which produced a minimum ratio at A = 0. In 
the second, it was Ax7 = —1.25. 


Progress through Changing of Bases 


Computations continued in Table 5.7 as if there were nothing troublesome about 
A = 0. This is the normal way to plosue 


5.6 Degeneracy and Zero-Length Simplex Steps 


At t = 2 in Table 5.7, such perseverance was rewarded. Solution x” is just as 
degenerate as earlier x and x™, but real progress was made with a step of ) = 25. 

In virtually all practical linear programs, eventual objective function progress 
will occur in this way (see Section 5.7 for exceptions). The reason is a much more 
subtle form of progress going on as we change bases at each iteration. 


$39 | ‘Simplex compan will normally escape a sequence of degenerate 


also changes ance directions, eventually p 


a direction alone which 


positive progress can be achieved. - 


The sequence of simplex directions for x; is illustrated in Table 5.7. Att = 1 
that simplex direction was not even improving. But after a change of basis at ¢ = 2, 


the direction for x; is completely different. As a result it now improves without 
decreasing a zero-valued basis variable, and a positive step is possible. 


G WITH ZERO STEPS 


Genser the standard-form linear program 


max —xX, + 2x2 
s.t. xy — xX. +x3=0 
Kier, ware O 


(a) Begin with x3 basic and apply rudimentary simplex Algorithm 5A to demonstrate 
that the first iteration produces a move with step 4 = 0. 


(b) Continue for another iteration to demonstrate that a positive (in fact, infinite) 
step A is achieved at the second step. 


(c) Explain how the change of basis the first iteration contributed to this progress. 


Analysis: Simplex Algorithm 5A proceeds as follows: 


xy X> x3 
max ¢ -1 2, 0 b 
A 1 -1 -1 0 
t=0 N N 
x00) 0 0 0 ec xO= 0 
Ax for x 1 0 1 =-l 
Ax for x» 0 1 -1 e =[2] 
0 2 
iia. Wa 4nseeDlaon eo 
a N N 
xf) 0 0 0 e-x%=0 
Ax for x 1 il 0 aay =] 
Ax for x3 0 -1 1 G=- 
= — as “anbounded” 


_ (a) The only improving simplex direction at t = 0 decreases basic x3, which is already 
| at degenerate value =0. A step 4 = 0 results. 
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(b) After the change of basis at t = 1, the direction for x, now improves. It decreases 
no component of the solution, so 2 = +00 is possible. The model is unbounded. 


(c) Progress became possible because the change of basis produced a different sim- 
plex direction for x,. 


§ "7 CONVERGENCE AND CYCLING WITH SIMPLEX 


A numerical search algorithm is said to converge if iterations make steady progress 
toward a solution. It converges finitely if it is guaranteed to stop after a finite number 
of iterations. 


Finite Convergence with Positive Steps 
Does simplex converge? Yes, if there is progress at each iteration. 


With two phases we can also detect infeasibility. 

To see why this is true, recall that every iteration of the simplex algorithm pro- 
duces a basic feasible solution (i.e., one generated by a selected set of basic columns). 
If the model in standard form has constraint matrix A with m rows and n columns. 
there are only finitely many possible bases. Each chooses m columns from the list of 
nas we did in Table 5.1. This limit gives 

maximum number of bases = nee eS 
m!(n — my)! 
where k! 4 k(k —1),..., (1). Certainly, this is a large number, but it is still finite. 

When each step size i is positive, no basis can ever repeat. Each time, we 
pursue a simplex direction that strictly improves the objective. Thus there is no way 
we could return to a previous one with poorer objective value. 


ER OF SIMPLEX ITERATIONS 


A standard-form linear program has 10 variables and 7 main constraints. Assuming 
step size A > 0 at every step, compute a finite bound on the number of iterations that 
Algorithm 5A might require before terminating. 


Analysis: If there is positive progress at each iteration, we need only bound the 
number of possible bases. Each would have 7 basic variables selected from among 
the 10 available. Thus there can be at most 
n! eel 2 326285800 
mi(n—m)! 713! 5040(6) 
iterations before simplex computation must stop. 


= 120 


5.8 Doing It Efficiently: Revised Simplex 


Degeneracy and Cycling 


Degenerate linear programs (definition | 5.35 |) pose another threat to convergence 


of the simplex method. We saw in Section 5.6 (principle 
sometimes lead to simplex steps with A = 0. 

The concern raised by such degenerate moves of simplex search is that since no 
progress is being achieved in the objective function, a sequence of iterations might 
return to a basis we have already visited. Thereafter, the cycle would repeat, and the 
simplex would never stop. 

Cycling can occur. Table 5.8 shows a carefully crafted example. For brevity, 
the table shows simplex directions only if they were actually used, but each is clearly 
an improving direction. Simplex passed through the degenerate basis sequence 


7 |) that degeneracy can 


X1, X42, %3 
X25 X35 X4 
X3, X4, X5 
X3, X5, X6 
X35 X6, ¥7 
x1, X3, X7 
X1, X2, X43 
starting and ending on the same basic set. 
Despite examples like the one in Table 5.8, cycling is quite rare. In fact, small 
examples of LPs that cycle are so rare that the one in Table 5.8 is rather famous. 
For the overwhelming majority of LP models it is safe to proceed just as set out 
in principle | 5.38}. If a step A happens to =0, act as if it were small but positive, and 
continue the search. 


5.41 | It is usually safe to assume that cycling will not oceur in applied linear 
progr mming models and thus that ‘Simplex se searc 


In those rare cases where cycling is a threat, careful choice of entering and 
leaving basic variables can restore simplex convergence. However, details of such 
anticycling rules are beyond the scope of this book. 


5.8 DOING IT EFFICIENTLY: REVISED SIMPLEX 


The approach we have taken to simplex search so far emphasizes the underlying logic 
of the algorithm. However, many of the steps can be executed much more efficiently 
with the aid of a bit of matrix algebra. In this section we outline the revised simplex 
algorithm which is at the heart of large-scale codes. 


Computations with Basis Inverses 


Rudimentary Algorithm 5A of Section 4.3 solves a great many systems of linear 
equations. We must solve one such system to find the first basic solution. Then at 
each iteration a linear system is solved for every simplex direction. 
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TABLE 5.8 Example of Cycling with Simplex 


xy X2 x3 x4 X5 X6 x7 
mine 0 0 0 ~0.75 20 ~0.50 6 b 
1 0 0 0.25 = =I 9 0 
A 0 1 0 0.50 a1 ~0.50 3 0 
0 0 1 0 0 1 0 1 
t=0 B B B N N N N 
x() 0 0 1 0 0 0 0 cx = 0 
Ax for x, 0.25 ~0.50 0 1 0 0 0 Z, =E0.75 
0 _0O| = 
v5] 050 pies 
t= N B B B N N N 
x) 0 0 1 0 0 0 0 ex =0) 
Ax for x5 0 4 0 32 1 0 0 é5 =|-4.0 
as 0 aS =: = 
eo 
t=2 N N B B B N N 
x?) 0 0 1 0 0 0 0 ex) =0 
Ax for x¢ 0 0 -1 -8 -0.38 it 0 C= 
ar, ee fl nOe pie a 2 
1 0.38 wae 
=3 N N B N B B N 
x) 0 0 1 0 0 0 0 ex) =0 
Ax for x7 0 0 -10.5 0 0.19 10.5 1 C= 
et we als = oe = Ba f 
105 0.19 ao 
1=4 N N B N N B N 
x4) 0 0 1 0 0 0 0 ex =0 
Ax for x; 1 0 2 0 0 a) ~0.33 z, =EL0 
a ts = es ee 0 £00 EO 
0.33 
=5 B N B N N N N 
x0) 0 0 1 0 0 0 0 ex) =0 
Ax for x, 3 1 0 0 0 0 -0.33 @ =E2.0 
0 
= at or yes 
0.33 
=6 B B B N N N 
x 0 0 1 0 0 0 0 cx =0 


The first insight leading to computational efficiency is to realize that the linear 
systems for any iteration have the same left-hand side. All involve solving for weights 
on basic columns to express another vector. 

Consider, for example, iteration t = 1 of the Top Brass computation detailed in 


Table 5.2. With all nonbasic variables fixed =0 (definition ), the current basic 
solution can be completed by solving the linear system 


tix, +0x, +0x5 +0x, = 1000 
+0x, +1x, +0x5 +0x, = 1500 
tix, +0x, +1x5 +0x6 = 1750 


+4x, +0xg +0x5 +1%5 = 4800 


5.8 Doing It Efficiently: Revised Simplex 


Simplex directions are similar. Definition implicitly requires solving for 
a representation of the negative of the constraint column for x; as we derive the 
corresponding simplex direction. For example, basic parts of such simplex directions 
Ax for Top Brass iteration t = 1 all come from linear equation systems 


+1 Ax, +0Ax, +0Ax5° +0Ax%6 = -a; 
+0 Ax, +1Axq, +0Ax5 +0Ax56 = by 
+1 Ax, +0Ax, +1Ax5 +0Ax5 = =a3,; 
+4Ax, +0Axq +0Ax5 +1Axe = —@4; 


Think of the collection of columns for basic variables as a basis matrix: 


All needed linear systems for this B can be solved easily if we compute just once the 
inverse of matrix B: 


PAOEORe0 
oe lage Ore pal 
Be oo 

apage gm 


Primer 5 reviews the key facts about matrix inverses. For our purposes the im- 
portant observation is that multiplication by a basis inverse can solve a corresponding 
system of linear constraints. 


5.42 Using a representation of the current basis matrix inverse B™ | basic com- 
ponents of the corresponding basic solution can be computed by matrix mul- 
tiplication: as B-'b, and basic components of simplex directions for nonbasic 
variables x; are -B ‘a, where b is the vector of ge side coefficients, 
and ai isthe constraint column forxj 


We can illustrate with the basis inverse above. Basic components of the corre- 
sponding basic solution can be computed (formula ): 


x1 105 0.20 1000 1000 
rad eet tae feta nda 1500 | _ [| 1500 
oo| Sab 40) Steo 17501) = |. 750 
X6 2A 0 HOt 4800 800 


Similarly, basic components of the simplex direction for x2 are 


fs Perret a 0 
1c Sa te pe ait. ie a fe 
SC "eax cic acta |e leas aes oa iia ages aos 
Nie ea dhe et le a se) 55) 
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_ PRIMERS: 


ie any) ers Aca any vector x. 
are those of the form | 


Matrix M7 is called thet oS . Mand denoted by a —1 expone 
that are not square or are singular (see Primer 4) have no inverses 
As an —s vec inverse of fhe matrix. 


a 
5 
T6 

7 

6 


MAAS 


= (Continued) 


5.8 Doing It Efficiently: Revised Simplex 


Consider the standard-form linear program 


min 9x, +3x2 +1%4 
st. 2x, +1x%. —1x3 =12 
1x1 +9x3 +2x4 =5 


X1,%X2,%X3,X4 = 0 
and assume that x; and x2 are basic. 


(a) Identify the current basis matrix. 
(b) Compute the current basis inverse. 
(c) Use your basis inverse to compute the current basic solution. 


(d) Use the basis matrix inverse to compute all simplex directions at the current 
solution. 


Analysis: 
(a) Columns for basic variables x; and x2 make the basis matrix 
eral 
B=(i 0) 
(b) Applying the formula in Primer 5 yields 


pile 1 ( 0 Palen al 
5 OO) Ode Sh se yi V1 i =2 


(c) Using computation 
are 


, the components of the basic solution for basic variables 


fro, Ral ay) se) nh >) 


Thus the full basic solution is x = (5, 2, 0, 0). 
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, the basic components of the direction for x3 can be 
computed: 


AG AiG) 0 1 iI \ 2/7 =9 
(etn Sait ee Sg See 
Thus the full simplex direction is x = (—9, 19, 1, 0). Basic components of the corre- 
sponding direction for nonbasic x4 are 


Canyon ($ 2)(8)=(2) 


which implies a full simplex direction x = (—2, 4, 0, 1). 


Updating the Representation of B“ 


It would require much too much memory to keep handy a copy of the full basis 
inverse matrix for most real linear programs. Still, production-quality codes do 
employ a representation of B~! in much the same way as we have just outlined to 
speed up the solution of required linear systems. 

Recall that simplex changes the basis at each iteration. If a new basis matrix in- 
verse (representation) had to be computed each time, some of the work we did in Sec- 
tions 5.3 and 5.5 would be reduced, but an immense amount of calculation would 
remain. 

The next insight for reducing simplex computation is that bases differ very lit- 
tle from iteration to iteration. Corresponding basis matrices are not very different 
either. Exactly one column of the old B (the one for the leaving basic variable) 
is replaced by one new one (the column of the entering nonbasic variable); other 
columns remain unchanged. 

Fortunately, we can exploit this similarity of bases. 


ey B) = ‘eo Hee where Ei is an update matrix x constructed from 
chosen simplex direction. _ 


That is, the inverse can be updated by a single matrix multiplication. The process is 
called a pivot. 
The specific form of update matrices E is indicated by the schema 


leaving position \, 


Axis 

f) Qleaua) Sete Qi RO 
AXjeave 
AXx2n 

Onaee ag) poe, foul stg 
AXleave 

E= ie oT G 0 ee) 

1 
AXeave 

0 0 0 0 

0 0 0 — Atmth 1 


AXleave 
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Here Ax; denotes the simplex direction component for the jth basic variable, and 
AXjeave indicates the component for the leaving basic. The matrix is almost an iden- 
tity. Only the column position of the leaving basic variable is replaced. The jth 
row of that special column is — Axjth/ AXicave, except on the diagonal, where it is 
sly AXteave- 

Again using iteration ¢ = 1 of Table 5.2, the required pivot replaces x¢ in the 
fourth position of the old basis {x1, x4, x5, x6} by x2. The corresponding E matrix is 


0 

1 960 sine 

9 

6 3 aA 
By ats 
(7 10s es 

a5 

1 

0): 10: tae 

=) 


The basis inverse after the iteration can then be computed from the old B~! above 


as (pivot formula ) 


POOH 
OL ei 
-l aA 
new B f= HOMO 
ANNO wO? 2 
= E(oldB™! 
0 
£ Ge 
ee) t 6-0 @ 
si | 
i Obs neck aa) Oued rd: 
Te a ee tae a ol | cee eae 
ey 
ae 1 4.) O° 1 
: 5 
dae aly cect Recaal) 
7 Deer ley aa a 
shy iL. 10> i as 
2D) cy fount calles 


Although it is beyond the scope of this book to provide details, update formula 
$43 | also hints at how production simplex computer codes actually represent basis 
verses. Since each inverse is just E times the last, we could represent the current 

sverse simply by remembering its initial form along with the E’s from iterations 
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so far. Then the process of multiplying a vector by B~!, required, say, to compute 
a simplex direction, can be accomplished just as well by multiplying in turn by the 
initial form and all the E’s. 


SAMPLE EXERCISE 5.24: UPDATING Basis INVERSES _ 


Assume in the linear program of Sample Exercise 5.23 that nonbasic x3 enters the 
basis and x; leaves. Use the simplex direction results of part (d) to compute update 
matrix E, and apply the result to compute the new basis inverse. 

Analysis: From Sample Exercise 5.23(d), the basic components of the simplex di- 
rection for x3 are Ax; = —9 and Axp = 19. With x; leaving the basic set {x1, x2}, 
schema (5.18) gives : 


1 
—-— 0 1 9g 
ree Ax, cx 9 
ee 1 i 1 
Ax, 


Using this E and the old basis inverse of Sample Exercise 5.23(b), updating 


gives 


Let) 0 Al 0 
= St ie 9 os 
new B= E(old B = ae 4)=() 


Basic Variable Sequence in Revised Simplex 


One slightly confusing aspect of equation (5.18) is its reference to basic variables by 
sequential position (1st, 2nd, etc.) rather than the original subscript. For example, 
the pivot above for Top Brass iteration t = 1 entered x2 to replace x¢ as the 4th basic 
variable. The new basis thus has 


ole OlR 
_— 


A A 
Mies een da Gr Ney - Kain AX 


and columns are arranged in that order in the foregoing expressions. 


_5.44 | Variables in revised simplex enter the b. 
tions as the variables they replace. : - 


This sequence-based numbering of basis variables recurs in several elements 
of revised simplex computation. Readers doing calculations by hand will find it 
somewhat tedious. Keep in mind, however, that revised simplex ideas are designed 
for efficient computer calculation, not ease of human understanding. 


SAMPLE EXERCISE 5.25: TRACKING BASIC VARIABLE SEQUENCE 


Assume that revised simplex computation on a linear program in standard form 
begins with basic variable sequence {x;, x2, x3}. On subsequent pivots x6 replaces x1, 
x5 replaces x3, and x4 replaces x5. Show the basic variable sequences for those three 
iterations. 
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Analysis: The initial basis has 
ig is Xond AM, ra 2H 
As variables enter and leave, it changes to 
Rise S Xe; Xond SPS Nara S ¥s 
Xist 2X6, Xana SX, Xara A Xs 


A A Nee 
Xist = 4, X2nd = *2, X3rd = X5 


Computing Reduced Costs by Pricing 
Simplex search chooses a move direction (or concludes optimality) by computing the 
reduced cost of definition | 5.22 | for nonbasic variables x;, that is, 


nN 
Gj Aec-Ax a :3 Cr AXg 
k=1 
where Ax is the simplex direction for nonbasic x;. 

The only reason for computing most of the simplex directions at any iteration 
is to determine reduced costs. We actually use just one simplex direction in the 
move. Obviously, a great deal of computation could be saved if we could obtain the 
c; without generating full simplex directions. 

To see exactly how to do that, recall that the simplex direction Ax for nonbasic x; 
is =1 in the jth component and =0 on components for all other nonbasics (definition 


5.21 |). Thus we could write 
Gat >) cy Axe (5.19) 
keB 
where B denotes the set of basic variable subscripts. For example, in iteration f = 1 
of the Top Brass example, 
c3 = 0+ [12(—1) + 000) + 0(1) + 0(4)] = —12 

We now know how to determine the basic directional components Ax, in ex- 

pression (5.19) using a representation of the basis inverse (formula Ds 


AXxist a1, 

Axon shy G2 
= B g 

AX mth —am,k 


where again Ax;, is the component of simplex direction Ax on the jth basic variable. 
Denoting corresponding basic objective function coefficients by cin and substituting 
oroduces the pricing vector for the current iteration. 


5.45_| The pricing 
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Then, using formula , equation (5.19) becomes 


AXist 
c AXxond 
Cj = Cj st (Cist Condiesacos Cmth) e 
AXinth 
aj 
=) 42,5 
= Cj — (Cist, Conds -- +» Cmth)B 
am,j 
nj 
a2j 
=cCj-V: 


We see that reduced costs ¢; can be computed without explicitly generating 
simplex directions. 


the Sone constraint matrix column for variable 3 Xe 


Again we illustrate with iteration t = 1 of Table 5.2. The pricing vector for that 
iteration is 


ieee) x00) 
V = (Cist, Cond, «+», Cmth) BT! = (12, 0, 0,.0) s ; : ; = (12, 0,0, 0) 
-4 0 0 1 
Then reduced costs follow as : 
; 
@ =c—v-a? =9-—(12,0,0,0)- : =9 
2 
1 
@ =c3 —v-a® =0—(12,0,0,0)- ; =-—12 
0 


Formula| 5.46 | changes absolutely nothing about the value of the reduced costs. 
but it decreases enormously the effort to compute them. We now need to compute 
only one simplex direction explicitly—the one that the nonbasic variable was chosen 
to enter. 
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Consider the linear program 


min 3x, +100x. +12x3 —8x4 

s.t. 3x4 + 1x — 1x3 = 90 
—-lx, -1xy lean 22 
X1,X2, X3,.X4 > 0 


Assume that the current basic variable sequence is {x3, x1}. 
(a) Compute the corresponding basis inverse. 
(b) Compute the associated pricing vector. 


(c) Compute the reduced costs on nonbasic variables without generating any simplex 
directions. 


Analysis: 


a) In the sequence given, the basis inverse matrix is 


-1 
ig) tat obey 3 Lope! oad 
cod i ( Caer h tO =! 
5) Applying definition , the associated pricing vector is 
oy -1 -3 
v= (Cist; Cond) B = (12, 3) Oe = CZ, —39) 
2) We can now compute all reduced costs via principle y 


@& = c—v-a® = 100- (—12, —39)- (1, —1) 73 
Gf = c—vy-a® = —8- (-12, —39)- (0,1) = 31 


Il 


Revised Simplex Search of Top Brass Example 


ve now have all the pieces of the revised simplex procedure detailed in Algorithm 

SC. Table 5.9 shows the complete revised simplex search of the Top Brass Trophy 

-sample done earlier in Table 5.2. It is important to note that the sequence of basic 

‘casible solutions encountered is exactly the same in the two tables. Only the method 
* computation at intermediate steps has changed. 

To avoid confusion about sequential numbering of basis variables, bases in 
Table 5.9 are marked with their positions. For example, the basis to start iteration 

= 3 has x, as the first basic, x2 as the fourth basic, x3 as the third basic, and x4 as 
>< second basic. The variables x5 and xs are nonbasic. 

In Table 5.9 we maintain an updated representation of the basis matrix inverse at 
-schiteration. That of iteration t = 0 is an identity matrix because the corresponding 
B ss an identity, and it is easy to check that the inverse of an identity matrix is an 

Sentity matrix. 
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i i ¥ D } : AD AD JEAD DRDOGDA) 


Step 0: Initialization. Choose any starting feasible basis, and construct a 
representation of the corresponding basic column matrix inverse B~'. Then 
use that representation to solve Bx? = b for basic components of the initial 


solution x, set all nonbasic ie < 0, and initialize solution index ¢ — 0. 


Step 1: Pricing. Use the representation of the current basic column inverse 
B=! to solve vB = ce? for pricing vector v where c? is the vector of basic 
objective function coefficients. Then evaluate reduced costs @ <— cj —v-a” 
for each nonbasic x;. 

Step 2: Optimality. If no c > 0 for a maximize problem (or no G < 0 
for a minimize), then stop; current solution x is globally optimal. Otherwise, 
choose an entering nonbasic variable x, with improving Cp. 

Step 3: Simplex Direction. Construct the simplex direction Ax“ for 
nonbasic variable x, using the representation of current basic column inverse 
B“! to solve BAx = —a? for basic components. 

Step 4: Step Size. If there is no limit on feasible moves in simplex direction 
Ax“ (all components are nonnegative), stop; the given model is unbound- 
ed. Otherwise, choose leaving variable x, so that 


x0 x0 i 
aa = min} —_ aay fee 0 andset A<— = San 
Beau: =o — Ax; 


Step 5: New Point and Basis. Compute the new solution 
x@tbD — x ae AAx tt) 


and replace x, in the basis by x,. Also construct the associated pivot matrix 
E and update the basis inverse representation as EB-'. Then advance 
t <¢+1, and return to Step 1. 


The next three iterations update B~! via formula . Required E matrices 


are, respectively, 


pit a) Lo 0. bd Pasi 
a 0. 6 = Ue eea 0 
0 = 2 
aed) Ain® 20 1 ces ple 
os 0 0 = Ord ie 0 
i < i , and ‘i 
pare (ON Oot they rice 
= = erties 0 
—4 it > 
eee | 0 oes: 2d © al 
eI 0 0 = 0 0 5 


At each iteration of revised simplex, reduced costs ¢; are computed directly 
using pricing vector v. Table 5.9 shows all v’s, and the resulting ¢;, boxing the one 
selected as improving at each iteration. The simplex direction associated with that 
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TABLE 5.9 Revised Simplex Search of Top Brass Trophy Example 


xy X2 x3 X4 X5 X6 
maxe 12 9 0 0 0 0 b 
1 0 1 0 0 0 1000 
0 1 0 1 0 0 1500 
A 1 1 0 0 1 0 1750 
4 2 0 0 0 1 4800 
t=0 N N Ist 2nd 3rd 4th 
x) 0 0 1000 1500 ~—-1750 4800 ee x =0 
hy OO pe0 0 
pio 5 2 9], v{¢ 
(OMNOMN Oe! 0 
Gj 12 9 0 0 0 0 
Ax for x; 1 0 -1 0 -1 —4 
1000 1750 4800 
Il) =Cl) 4 = 1000 
t=1 Ist N N 2nd 3rd 4th 
x(l) 1000 0 0 1500750 800 e+ x = 12,000 
1910 DRO 40 12 
B= ot oly Lo 
4 OmgOleul 0 
Gj 0 [9] -12 0 0 0 
Ax for x» 0 1 0 -1 -1 —2 
irs Se fi aah ay A = 400 
=2 Ist 4th N 2nd 3rd 
x?) 1000 400 0 1100 350 0 e+ x) = 15,600 
1 OAD (ODIO 6 
ee Ae ee dese A 
=D, 10 eee Op OS 4.5 
Gj 0 0 6 0 0 4.5 
Axforx,  —-1 2 1 5} = 0 
aa) 43 ES) = 
G3 1st 4th 3rd 2nd N N 
x0) 650 1100 350 400 0 0 e+ x@) = 17,700 
0-0 05 Oe 
wo 5 Ta]. 18 
0 Dea 2. 205 1.5 
G 0 0 0 0 6 = “optimal” 


mproving reduced cost, which follows immediately in the table, is the only one 
explicitly generated at each iteration. 
Once a move direction has been selected, the revised simplex makes the step 


“ze 2 decision exactly as in the earlier simplex algorithm. A new basic solution 
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x“+D is computed, a leaving variable is selected, the basis inverse representation is 
updated, and processing continues. 


SIMPLEX WITH SIMPLE UPPER AND LOWER BOUNDS 


S, 


In Section 5.1 we showed how LP standard form converts all inequalities of a problem 
to nonnegativity constraints x; > 0. In later sections we demonstrated how easy this 
form makes construction of basic solutions, simplex directions, and so on. 

Almost all those simplifications would follow just as well if we allowed inequal- 
ities to take the slightly more general simple lower-bound form 


xj 2G 
or simple upper-bound form 
sae 


Here the @; and wu; are given model constants. Nonnegativities are just the special 
case of simple lower bounds with ¢; = 0. In this section we explore briefly how the 
revised simplex method of Section 5.8 can be modified to encompass simple lower 
and upper bounds. 


Lower and Upper-Bounded Standard Form 
The standard form for linear programs with simple lower and upper bounds is 


n 


min (or max) > CiXy 
j=l 
n 


s.t. Le a4 jx; = Dj for alli=1,2,...,m 
j=l 
Uy > xj = &; for allj =1,2,...,” 


Collecting lower and upper bounds in vectors € and u produces the corresponding 
matrix form. 


| 5.47 | Inmatrix notation, the lower and upper-bounded standard form for linear 


a = 
ex 
_Ax=b 
u>x>e 
Feasibility requires that u; > ¢;,and we allow the possibility that uj = 00, or £; = —ox. 


or both. 

Often, allowing simple bounds in this way considerably reduces the number o: 
rows m in main constraints Ax = b. The result is a savings in most of the complex 
steps of the simplex algorithm, such as generating simplex directions and updating 
B-!. For example, the Top Brass model of Figure 5.5 required that m = 4 rows in 
Tables 5.2 and 5.9; it needs only that m = 2 in lower or upper-bounded form. 


5.9 Simplex with Simple Upper and Lower Bounds 


xy Xo Xs X6 
max ¢ 12 i} 0 0 b 
il 1 1 0 1750 
4 2 0 1 4800 
£ 0 0 0 0 
u 1000 1500 oo co 


Notice that only two slack variables are now required. We continue to call them xs 
and x6, for consistency with Figure 5.5. 


PER-BOUNDED 


Place the following fia program in force and fone ebennced siniteanl form. 


min 3x, —x2+x3+ 11%, 
st. Xy +x. +2x3 +44 = 50 
x < 30 
3x1 + x4 < 90 
9x3 +x4 = 3 
Xz < 10 
x3 > —2 


X1,X2 = 0 


peat ysis: Nonnegative slack variables xs and x6 are required in the third and fourth 
nstraints, but the rest of the model, including unrestricted variable x4, can be 


accommodated by lower- and upper-bounded standard form . A full coefficient 
array is 


%y  %2 XG Xs 6 
min e¢ 3n, 1 tp ealal 0 0 b 
il 1 1 1 0 0 50 
3} 0 0 ih 1 0 90 
0 0 9 1 0 -l 5 
0 0, =2) co, 0 0 
a I) ee Co CO co 


Basic Solutions with Lower and Upper Bounds 


The main idea of basic solutions is to make active the inequalities for all nonbasic 
siables, then solve for the basics. With lower and upper bounds there may be 
-hoices for each nonbasic x;, depending on which of ¢; and uj are finite. 
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The solution is basic feasible if computed basic values satisfy their lower and upper 
bound constraints 
ujaxj = for all j basic 
The linear system to be solved in computing basic variable values has the form 
Sm ax, =b;- Ys agli += aj jUj for alli=1,...,m (5.20) 
jeB jeL jeU 
where L indexes the set of lower-bounded nonbasics and U the set of upper-bounded 


nonbasics. Except for the more complicated right-hand side, it is no more difficult 
to solve than that of the ordinary simplex. 


Grasides the linear program 
min —2x, +4x. +3x3 
s.t. Ly —1x2 +3x4 = 13 
2x4 tix. +2x3 —2x4 
62 ap> 4, 10S x eS" Sexy Sl, (3 2 xe 2 


ll 


Compute the basic solution having x2 and x3 basic, x; nonbasic lower-bounded, and 
x4 nonbasic upper-bounded. 


Analysis: Following definition , nonbasic variables will be fixed: 
y= =4 and KAS =3 
We solve for basic components in equations (5.20): 


—1lx. +0x3 = 13 —()4-(@)@) 
41x. +2x3 = 6 —(2)(4) — (—2)(3) 


The unique solution is x. = 0, x3 = 2, implying a full basic solution of x = (4, 0, 2, 3). 


ll Il 
SS 


Unrestricted Variables with No Bounds 


Sometimes models contain unrestricted variables that can take on any value. Ex- 
amples include inventory levels where backordering is allowed, and temperatures. 
In such cases £; = —co and uj = oo. With neither finite, the variables cannot be 


nonbasic under definition |. : 


5.49 | Unrestricted vacanies must be 
basic solution. — 


Increasing and Decreasing Nonbasic Variable Values 


Simplex directions Ax are constructed to be feasible when nonbasic x; are increased. 
The jth directional component is Ax; = +1. 


ee 


5.9 Simplex with Simple Upper and Lower Bounds 


This logic works fine for lower-bounded nonbasic variables because the only 


feasible move is an increase. Tests for improvement are as in principle | 5.23 |. 


inimize problem ita, a - 6. : 


The new element in lower- and upper-bounded simplex comprises upper- 
bounded nonbasic variables x;. If they are to change values and stay feasible, they 
must decrease (i.e., the jth directional component of a move must be negative). 

A little review of the derivations in Section 4.3 will show that the direction 
decreasing nonbasic x;, changing no other nonbasic, and maintaining Ax = b is 
precisely the negative of simplex direction increasing xj. Improvement tests are 
similarly reversed. 


5.51 | : ) -Ax for upper- -bounded nonbasie xi improves 
in a maximize problem if ¢ < 0 and in a minimize problem 


SAMP EXERCISE Ss, an TESTING | DIRECTIONS IN LOWER AND Upp 


Return to the liters program of ESanble Exercise 5.28. Compute eainedihe costs for all 
nonbasic variables, and determine which could enter to improve the objective value. 


Analysis: Basis matrix 


B= a has inverse Bo! = = 
Ih 5 5 
pavaD 


Thus, pricing vector 
v=c'B" = (5,3) ( 7 
2 
Then, applying computation yields 


a = cy —vea = Spel a y4G) 2) oe yf ae 
f@ = c-vea® = 0-(-},3):6,-2) = F 


Nie 


These reduced costs both qualify as improving under conditions | 5.50 | and : 


Step Size with Increasing and Decreasing Values 


Step size rule assumes that nonbasic variables always increase and that the 
lower bound of every basic variable is zero. With a lower- and upper-bounded sim- 
plex, many more possibilities exist. If a basic variable is decreasing from value x,, 
the maximum feasible decrease is now (x, — £x). If a basic variable is increasing from 
value xx, the maximum feasible increase is (uz — xx). Also, the changing nonbasic 
variable may itself fix 1 because it can feasibly increase or decrease only (uz — x). 
These cases lead to a more complex step size rule. 
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Return again to the example of Sample Exercise 5.28. Sample Exercise 5.29 estab- 
lished that both nonbasic variables qualify to enter. Determine the corresponding 
stepsize and leaving variable for each. 


Analysis: Using the results of Sample Exercise 5.29, basic components of the simplex 
directions for x; are 


cass} AE cle fuga) ashe) 


The full direction is Ax = (1, 1, -3, 0). With increasing orientation 6 = +1, rule 


gives 


4 . {2-1 2 
xX” <— min 3 = 


At = min |" =} 


ge Seta | 
A < min{2, 2} = 2 
The leaving variable, which set the value of i, is the basic x3. 
Basic components of the simplex directions for x4 are 


Ni ieee temaliote lee ear 


making the full direction Ax = (0, 3, —}, 1). With decreasing orientation 6 = —1, 


rule gives 


Wa min fo 2 )e! 


3 ae 


Ab oe nin | 557] = 


A << min{l1,6}=1 
This time the leaving variable is the nonbasic xz itself. 


5.9 Simplex with Simple Upper and Lower Bounds 


Case with No Basis Change 


The third possibility above, where i is established by the changing nonbasic itself, 
presents another new issue. We have no leaving basic variable. In fact, 


vice versa. We ne 


ed not change — 


Lower- and Upper-Bounded Simplex Algorithm 


We are now ready to state the full lower- and upper-bounded revised simplex Algo- 
rithm 5D employed in most commercial computer codes. As in earlier sections, a"? 
denotes the (lower- and upper-bounded) standard-form constraint column for x;, and 
c8 4 (cist, Conds ---s Gmth) Fepresents the vector of basic variable objective function 
coefficients. 


Lower- and Upper-Bounded Simplex on Top Brass Example 


Table 5.10 returns one last time to the Top Brass Trophy example of Figure 5.5. 
It details the lower- and upper-bounded simplex computations paralleling those in 
Tables 5.2 and 5.9. 

With lower and upper bounds, only 2 basic variables must be chosen. We select 
the two slacks and make both x; and x2 nonbasic lower bounded. This produces 
starting basic feasible solution x© = (0, 0, 1750, 4800) with objective function value 
=0. The corresponding basis inverse B~' and pricing vector vy are shown in Table 
5.10. 

At iteration t = 0, both nonbasic lower-bounded variables have positive re- 
duced costs for this v. We choose the one for x;, making p = 1 and orientation 
5 = +1 to indicate increase. 

Next we compute step size ’ by rules [Be . Each unit step in that direction 
decreases x5 by 1 and x¢ by 4, so that 


1750 —0 4800 —0 


, = 1200 
1 + | 


A < min { 


The move also increases x; by 1 per unit step, making 


1000 — 0 
1 


None min | = 1000 


The maximum step feasible for both sorts of change is 
4 = min{A~, 47} = min{1200, 1000} = 1000 


with r = 1. 

A step of size A in the chosen simplex direction moves us to new solution 
x) = (1000, 0, 750, 800) at objective value 12,000. Since it was the changing nonbasic 
r, that fixed A (i.e., p =r), no modification of the basis is required (principle 5.53] ). 
The variable x, merely switches to nonbasic upper bounded. 
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Step 0: Initialization. Choose any starting feasible basis with initial nonbasic 
j € L at € and nonbasic j € U at uj, and construct a representation of the 
corresponding basis inverse B~'. Then use the representation of B-! to 


solve 
xs ai,jx,> = bj — = Gigl; — s agi; TOnalt =|)... 


jJEB jel jJeU 


for basic components j € B of initial solution x, and set solution index 
03 

Step 1: Pricing. Use the representation of the current basic column matrix 
inverse B-' to solve vB = c? for pricing vector v, where c? is the vector 
of basic objective function coefficients. Then evaluate reduced costs g <— 
cj — v-a? for each nonbasic x;. 


Jand[5.51]), 


stop; current solution x® is globally optimal. Otherwise, choose an entering 
nonbasic variable x, with improving Z, and set orientation 6 = +1 if the 
nonbasic is lower bounded and 5 = —1 if the nonbasic is upper bounded. 

Step 3: Simplex Direction. Construct the simplex direction Ax“*) for 
nonbasic variable x, using the representation of current basic column inverse 
B | to solve BAx = —6 a?) for basic components. 


Step 4: Step Size. Apply rule to determine the maximum feasible step 


A in direction Ax“*). If there is no limit (4 = oo}, stop; the given model 
is unbounded. Otherwise, choose as leaving x, any blocking variable that 


established the value of A in 552). 


Step 5: New Point and Basis. Compute the new solution 
xD 2 xO 4 15 Ax@tD 


If p #1, also replace x, in the basis by x,, construct the associated pivot 
matrix E, and update the basis inverse representation as EB~!. Then 
advance t <t +1, and return to Step 1. 


Iteration t = 1 proceeds in much the same way except that this time we consider 
both decreasing nonbasic x; and increasing x2. Only the second satisfies tests for 
improvement, so the Ax for x2 is pursued until the basic x6 drops to £5 = 0 at step 
i = 400. The new solution is x = (1000, 400, 350, 0), with objective value 15,600. 

At iteration ¢ = 2 we encounter a nonbasic decreasing from its upper bound. 
With c; = —6 < 0, the negative simplex direction for x2 helps our maximize objective 
( [Bt ). Orientation indicator 6 = —1. We step distance A = 350 in direction Ax to 
reach new point x® = (650, 1100, 0, 0) at value of 17,700. Iteration ¢t = 3 completes 
the computation, verifying that no available simplex move can improve the objective 
function. 
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Exercises 


TABLE 5.10 Upper- and Lower-Bounded Simplex 
Search of Top Brass Trophy Example 


ei 2 x5 x6 
max c 12 9 0 0 b 
1 1 1 0 1750 
4 2, 0 1 4800 
id 0 0 0 0 
u 1000 1500 —_— —_— 
t=0 1f IE Ist 2nd 
x) 0 0 1750 4800 c-xO=0 
FL aire 0 
Bas O<rithec Valo 
Gj 12 9 0 0 
Ax for x; 1 0 -1 4 
1000]|/SeeaL wal750% 204800 ie 
mae = yg A = 1000 
jill U L Ist 2nd 
x() 1000 0 750 800 e+ x“) = 12,000 
‘S(t 10 0 
B=(5 9). v=(0) 
Gj 12ilvie (3 0 0 
Ax for x (0) 1 =] =A 
1500 = 750 ~—« F800 . 
ut A = 400 
f=) U 2nd Ist L 
x?) 1000 400 350 0 e+ x) = 15,600 
1 -0.5 0 
= (ae be 
: at os} v-(23 
Z, 0 0 45 
Ax for x, -1 2 -1 0 
1000 =©1100 350 = 
1 5 i A = 350 
t=3 1st 2nd 1b, L 
x) 650 1100 0 0 e+ x°) = 17,700 
-l O05 6 
—1 _ ~~ 
y -| 2 Reh v=(°.] 
Gj 0 0 -6 -1.5 “optimal” 
EXERCISES 
5-1 Consider the linear constraints (b) [|] Determine geometrically whether each of 
ee the following solutions is infeasible, boundary, 
wee extreme, and/or interior: w” = (2,3), w® = 
WSS (0, 3), w® = (2,1), w® = G, 3). 
Wi, W2 = 0 : : 
(c) EQ] For those points of part (b) that are feasi- 


2) Sketch the feasible space in a 2-dimensional 


plot. 


ble, demonstrate algebraically whether they are 
boundary or interior. 
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(d) EQ] Determine for each of the points in part 
(b) whether a suitable (nonconstant) objec- 
tive function could make the point optimal or 
uniquely optimal. Explain. 


5-2 Do Exercise 5-1 for the LP 


3wy + Sw = 15 
5w, + 3w2 < 15 
W1,W2 >0 


and points w) = (0,0), w? = (1,1), w® = (2, 0), 
and w® = (3, 3). 

5-3 Place each of the following LPs in standard form 
and identify the corresponding A, b, and ¢ of defini- 


tion : 


(a) B) min 4x, + 2x) — 33x; 
s.t. x, — 4x3 +x3 < 12 
9x, + 6X3 =i5 
—5x1 + 9x, > 3 
Xi, Xa5eXBi=— 0 


(b) max 45x, + 15x3 
s.t. 4x, = 2x2 + 9x3 =122 
—2x, + 5x2 -x3>1 
Xi, —-X2< 5 
X1,%2,X%3 = 0 


(c) Bl] max 15(x; + 2x) + 11 (x2 — x3) 
s.t. 3x, > Xp +X. +-X3 


Os = 35 Paes 
(d) min 53x; +33(x, + 3x3) 
St ela = 1D 
Day = 10 
xj =0 (ee 


(ec) &] min 2x, +x. — 4x; 
St. xX, —X2 — 5x3 < 10 
3x2 + 9x, = —6 
x, > 0, x3 <0 
(f) max 4x, -—x 
s.t. —4x, — XX) + 7x3 > 9 
—X1 — xX. +3x3 < 14 
x, <0,%3>0 


5-4 Consider the linear constraints 
—yi +y2. <2 
Sy; < 10 
yi, y2 = 0 
(a) Sketch the feasible set in a 2-dimensional plot. 


(b) B] Add slacks y3 and yg to place constraints in 
LP standard form. 
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(c) [| Determine whether columns of standard 
form corresponding to each of the following 
sets of variables form a basis: {y,, y2}, {y2, y3}, 
{y3, va}, (v1, va}, {3}, {1 Yo, Ya}. 

(d) [| For each set that does form a basis in part 
(c), determine the corresponding basic solution 
and classify it as feasible on infeasible. 

(e) EQ] Identify each solution of part (d) on your 
plot of part (a), and comment on the connection 
between basic feasible solutions and extreme 
points. 


5-5 Do Exercise 5-4 for the LP 


yi +2y. <6 
y2 <2 
V1, y2 = 0 


and possible basic sets {y1, yo}, {v2, y3}, {yi}, {y2. ya}. 
{y2, 3, Ya}, (v1, Y3}- 

5-6 Write all conditions that a feasible direction Aw 
must satisfy, at the solution w indicated, to each of the 
following standard-form systems of LP constraints. 


(a) Rl 5wi+1w.-1w3=9 atw=(2,0,1) 
3w, — 4w2 + 8w3 = 14 
W1,W2,W3 > 0 

(b) 4w; —2w. +5w3=43 atw=(0,1,9) 


4w, + 2w2 — 3w3 = —25 
W1,W2,W3 = 0 


5-7 Following is a maximizing, standard-form linear 
program and a classification of variables as basic and 
nonbasic. 


x X> x3 X4 
maxec 10 1 0 0 b 
-1 1 4 Pik 13 
2 6 0 —2 2 

B N B N 


(a) [|] Compute the current basic solution. 

(b) [|] Compute all simplex directions available at 
the current basis. 

(c) [&] Verify that all your simplex directions are 
feasible at the current basic solution. 

(d) [Q] Determine whether each of the simplex di- 
rections is improving. 

(ec) ] Regardless of whether it improves, deter 
mine the maximum step that preserves feasibil- 
ity in each simplex direction and the new basis 
that would result after such a step. 


5-8 Do Exercise 5-7 for minimizing the standard- 
form LP 


eel che 3 x 
mince 8 -5 0 1 b 
13 2 3 1 7 
-4 1 0 -1 -l 
N N B B 
5-9 Consider the linear program 
max 3z,+ 22 
s.t. —271+%2 <2 
Z+2%2 <6 
z<4 
Z1,%2 =0 


(a) [Q] Solve the problem graphically. 

(b) &] Add slacks z3, z4, and z; to place the model 
in standard form. 

(c) [| Apply rudimentary simplex Algorithm 5A 
to compute an optimal solution to your standard 
form starting with all slacks basic. 

(d) [Q] Plot your progress in part (c) on the graph 
of part (a). 


5-10 Do Exercise 5-9 for the LP 


max 2z;+5z2 

st. 3z, +222 < 18 
PENS 5 
Z2 <3 
21,22 20 


5-11 The following plot shows several feasible points 
ina linear program and contours ofits objective func- 
tion. 
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Determine whether each of the following sequences 
of solutions could have been one followed by the 
simplex algorithm applied to the corresponding LP 
standard form. 


(a) &] P1,P8,P7,P6 
(b) P2,P4,P5,P6 
(c) [&] P1,P3,P6 
(d) P8,P7,P6 

(ce) [ P1,P7,P6 
(f) P4,P1,P8,P7,P6 


5-12 Construct the simplex dictionary form 
corresponding to each of the following. 


(a) [S| The model and basis shown in Exercise 5-7 
(b) The model and basis shown in Exercise 5-8 


5-13. Rudimentary simplex Algorithm 5A is being 
applied to optimize a linear program with objective 
function 


min 3w; + 11w2 — 8w3 


Determine whether each of the following simplex di- 
rections for w4 leads to a conclusion that the given 
LP in unbounded. 


(a) &] Aw = 1,0, —4, 1) 
(b) Aw = (1,3,0, 1) 

(c) &] Aw = (1,0, 3,1) 
(d) Aw = (1,1, -2, 1) 


5-14 Consider the linear program 


max 4y; + 5y2 

s.t. -yi ty2 <4 
Yi — yo < 10 
y1,92 20 


(a) Show graphically that the model is unbounded. 

(b) [|] Add slacks y3 and y4 to place the model in 
standard form. 

(c) [] Starting with all slacks basic, apply rudimen- 
tary simplex Algorithm 5A to establish that the 
original model is unbounded. 


5-15 Do Exercise 5-14 for the LP 


min —10y; + y2 

st.  —Sy; +3y2 < 15 
3y1 —Sy2 <8 
y1,y2 20 


5-16 Setup each of the following to begin Phase I of 
two-phase simplex Algorithm 5B. Also indicate the 
basic variables of the initial Phase I solution. 
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(a) El max 2wi+w+9w; 
St; wy tw. < 18 
—2w, +w3 = —2 
3w2 + 5w3 > 15 
W1, W2, W3 > 0 
(b) min 7w,; —9w2 + 4w3 


st. Ww, —18w2 <9 
W3+W4 > 14 
W, + W2 —2w3 —3w4 = 1 
W1, W2, W3, W4 > O 


5-17 Setup each of the models in Exercise 5-16 to 
begin a big-M solution using rudimentary simplex 
Algorithm 5A. Also indicate the basic variables of 
the initial solution. 


5-18 Consider the linear program 


max 9y; + yz 

s.t. —2y; + y2 >2 
y2 <1 
Vis y2 =O 


(a) Show graphically that the model is infeasible. 

(b) [] Add slacks and artificials y3,..., ys to setup 
the model for Phase I of Algorithm 5B. 

(c) [| Apply rudimentary simplex Algorithm 5A 
to this Phase I problem to establish that the orig- 
inal model is infeasible. 


5-19 Do Exercise 5-18 for the linear program 


min 2y, + 8y 

8.t. YMt+y2 2, 
vy = 3 
V1, y2 = 0 


5-20 Assuming that step size 4 > 0 at every step, 
compute a finite bound on the number of iterations 
of Algorithm 5A for each of the following standard- 
form linear programs. 


(a) [] The model in Exercise 5-7 

(b) The model in Exercise 5-8 

(c) &] A model with 1150 main constraints and 
2340 variables 

(d) A model with 345 main constraints and 11,236 
variables 


5-21 Rudimentary simplex Algorithm 5A is being 
applied to a standard-form linear program with vari- 
ables x;, ..., x5. Determine whether each of the fol- 
lowing basic solutions is degenerate for the given ba- 
sic variable set. 
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(a) ] B= {x1, x, x3},x = (1, 0,5, 0,0) 
(b) B= {x3, x4, x5}, x = (0,0, 1,4, 9) 
(c) | B= (4, x3, x5}, x = (1, 0,5, 0, 8) 
(d) B= {x,, x2, x4}, x = (1, 0, 0, 0, 0) 


5-22 Consider the linear program 


max x,+X2 

s.t. xX, +X2 <9 
—2x, +x <0 
Xx, — 2x2. <0 
X1,X > 0 


(a) [| Solve the problem graphically. 
(b) [8] Add slacks x3,.. 
standard form. 

(c) &] Apply rudimentary simplex Algorithm 5A 
to compute an optimal solution to your standard 
form starting with all slacks basic. 

(d) [& Plot your progress in part (c) on the graph 
of part (a). 

(e) How can the algorithm be making progress 
when A = 0 if some moves of part (c) left the 
solution unchanged? Explain. 


.,%5 to place the model in 


5-23 Do Exercise 5-22 for the LP 


max Xx, 

s.t. 6x1 + 3x2 < 18 
12x, — 3x, <0 
X1,X%2 >0 


5-24 Return to the LP of Exercise 5-7. 


(a) [|] Compute the basis matrix inverse corre- 
sponding to the basic variables indicated. 
(b) &] Compute the corresponding pricing vector 


of | 5.45]. 


(c) [Q] Without generating the implied simplex di- 
rections, use your pricing vector to determine 
whether each of them will be improving. 

(d) [| Foreachimproving simplex direction in part 
(c), generate the corresponding matrix E of 


3|, and compute the next basis matrix in- 


5-25 Do Exercise 5-24 for the LP of Exercise 5-8. 
5-26 Do Exercise 5-9 using revised simplex Algo- 
rithm SC in part (c). 

5-27 Do Exercise 5-10 using revised simplex Algo- 
rithm 5C in part (c). 


5-28 Suppose lower- and upper-bounded simplex Al- 
gorithm 5D is being applied to a problem with ob- 
jective function 


max 3x; — 4x. + x3 — 4x4 + 10x5 
3 main constraints, and bounds 


OSes 5 fall eee eo 

For cach of the following current basic solutions x 
and corresponding simplex directions Ax, then de- 
termine whether the appropriate move of + Ax is 
improving. Also compute the maximum step A that 
could be applied without losing feasibility and the 
basis status of variables that would result after such 
a step. Take the current basic variables to be those 
strictly between lower and upper bounds. 


(a) BI] x = (2,2,4,0,5), Ax = (1, —-1,0,0, 1) for 
x5 

(b) x = (5,0, 2,3, 2), Ax = (0, 1, 75. —#, 3) for x2 

(c) x = (0,1,0,4,2), Ax = (0, 0,1, —2, 2) for 
X3 

(d) x = (5,5, 1,3, 1), Ax = (1,0, 0, 4, 1) for x; 


5-29 Consider the linear program 


min 5z; + 6z2 
s.t. y+2>3 
3z, +272 >8 
0<z <6 
0<2 <5 
SUGGESTED READING 


Bazaraa, Mokhtar S., John J. Jarvis, and Hanif Sher- 
ali (1990), Linear Programming and Network Flows, 
Wiley, New York. 
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Exercises 


(a) EQ] Solve the problem graphically. 

(b) [Q] Add slacks z3 and z, to place the model in 
standard form for a lower- and upper-bounded 
simplex. 

(c) EQ] Apply lower- and upper-bounded simplex 
Algorithm 5D to compute an optimal solution 
to your standard form starting with all slacks ba- 
sic and original variables nonbasic at their upper 
bounds. 

(d) [| Plot your progress in part (c) on the graph 
of part (a). 


5-30 Do Exercise 5-29 on the LP 


max 62; + 8z2 

s.t. Z1+3z2 < 10 
Z+22 <5 
0<72<4 
0<z2<3 


Start with all original variables nonbasic lower- 
bounded. 


Chvatal, VaSek (1983), Linear Programming, W. H. 
Freeman, San Francisco. 


( H A P T f R 


Interior Point Methods 
for Linear Programming 


Although Chapter 5’s simplex method remains the most widely used algorithm for 
solving linear programs, a very different strategy emerged in operations research 
practice at the end of the 1980s. New interior point methods still follow the improving 
search paradigm for linear programs, but they employ moves quite different from 
those in the simplex method. Instead of staying on the boundary of the feasible 
region and passing from extreme point to extreme point, interior point methods 
proceed directly across the interior. 

Much more effort turns out to be required per move with interior point methods, 
but the number of moves decreases dramatically. In many large LPs, the result is a 
substantially shorter total solution time than any obtained so far with simplex. 

The first commercial interior point method for linear programming was N. Kar- 
markar’s projective transformation procedure, and developments have continued to 
this day. All methods are relatively complex mathematically, with many details be- 
yond the scope of an introductory book. 

In this chapter we seek to open up the exciting new interior point technology 
by concentrating on the broad strategies and intuitions behind the popular affine 
sealing and log barrier variants. In the final section we sketch some of the more 
advanced issues underlying commercial implementations. All development assumes 
reader familiarity with the search fundamentals of Chapter 3 and the LP conventions 
of Section 5.1. 


> 


6. | SEARCHING THROUGH THE INTERIOR 
Any linear programming search method allowed to pass through the interior of 
the feasible region poses a variety of new challenges. Before developing specific 
slgorithms, we introduce some of the key issues. 
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EXAMPLE 6.1: FRANNIE’S FIREWOOD 


To illustrate interior point computations, we need an example so small (and trivial) 
that it can be displayed graphically in a variety of ways. Frannie’s Firewood problem 
will serve. 

Each year Frannie sells up to 3 cords of firewood from her small woods. One 
potential customer has offered her $90 per half-cord and another, $150 per full cord. 
Our concern is how much Frannie should sell to each customer to maximize income, 
assuming that each will buy as much as he or she can. 

To begin, we need a decision variable for each customer. Define 


x, 4 number of half-cords sold to customer 1 
x2 4 number of cords sold to customer 2 
Then Frannie’s problem can be modeled as the linear program 
max 90x; + 150x, 
st. 5x1tx <3 (6.1) 
X1,X2 >0 


Figure 6.1 solves model (6.1) graphically. The unique solution sells all 3 cords to the 
first customer, making x} = 6, x} = 0. 


(1) — (50 53 
Sets 26° 26 


Vs. max 90x, + 150x, 


N 
es 
SS 


2 N 
ee x 


= (13) 


FIGURE 6.1 Graphic Solution of Frannie's Firewood 
Example 


Interior Points 


Recall (definition Ba, Section 5.1) that a feasible solution to a linear program is a 
boundary point if at least one inequality constraint of the model that can be strict 
for some feasible solutions is satisfied as an equality at the given point. A feasible 
solution is an interior point if no such inequalities are active. For example, solution 
x = (1, 5) of Figure 6.1 is an interior point because it satisfies all three constraints 
of model (6.1) as strict inequalities. 


Objective as a Move Direction 


There is one enormous convenience in searching from an interior point such as x 
of Figure 6.1 when all constraints are inequalities (we deal with equalities shortly). 
With no constraints active, every direction is feasible; that is, a small step in any 


——— te 


6.1 Searching through the Interior 


direction retains feasibility (see Section 3.2). Thus our only consideration in picking 
the next move is to find a direction improving the objective function. 

What direction improves the objective most rapidly? For linear programs it is 
easy to see that we should move with the gradient or objective coefficient vector as 


in principle (Section 3.3) . 


he  pbiective hinction= vector Ax = 


c. = noe 
to min e- <i Ar - 


For example, in the maximize Frannie model of Figure 6.1, we prefer Ax = ¢ = 
(90, 150) at x. This direction runs exactly perpendicular to contours of the objec- 
tive function. No alternative improves the objective faster. 


Each of cS Cowes is the objective function ote a eee program with 3 eon 
variables. Determine for each the direction of most rapid objective improvement. 


(a) min 4x, — 19x2 + x3 


(b) max —2x, — x2 + 79x3 


Analysis: We apply principle [6.1]. 
(a) For a minimize objective the direction of steepest improvement is 
Ax =-c= (—4, 19, —1) 


(b) For a maximize objective the direction of steepest improvement is 
Ax=c= (-2, -1,79) 


Boundary Strategy of Interior Point Methods 
Figure 6.1 shows the effect of using cost direction Ax = (90, 150) atx. A maximum 
feasible step of 1 = 73; would bring us to 
x) =x 4, Ax® 
= (1, 5) + 7590, 150) 


(50, 53) 
(46> 26 


Very rapid progress toward an optimum is achieved in a Single move. 
Notice that we would have to deal with the boundary at x. Constraint 


5X4 ae = 3 


is now active, and we discovered in Chapter 3 (principle 3.25|, Section 3.3) that 
feasible directions Ax at x must preserve this inequality by satisfying 


5 Axi + Ax, < 0 
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Such newly active constraints destroy the convenience of moves in the interior. 
That is why interior point algorithms stop short of the boundary as in the typical 
sequence plotted in Figure 6.2. Partial steps along suitable variations of the cost 


6.2 Interior point algorithms begin at and move through a sequence of inte- 
rior feasible solutions, converging to the boundary of the feasible region only 


atanoptimalsolution, = = 


SAMPLE EXERCISE 6.2: IDENTIFYING INTERIOR POINT TRAJECTORIES _ 


The following figure shows the Top Brass Trophy example from Section 5.1: 


x2 


2000 


(440, 530) 


Xe es 
e@ _ (4000, 400) 
4 j 
\(200, 300) °s 


s 
— 


Determine whether each of the following sequences of solutions could have resulted 


from an interior point search. 

(a) (0,0), (440,530), (650,1100) 

(b) (100,300), (440,530), (650,1100) 

(c) (100,300), (1000,400), (650,1100) 

(d) (0,0), (1000,0), (1000,400), (650,1100) 


—~ optimal x* = (650, 1100) 


6.1. Searching through the Interior 


FIGURE 6.2 Typical Interior Point Search of 
Frannie's Firewood Example 


Analysis: The sequence of solutions visited by an interior point search should con- 
form to principle : 
(a) This sequence is not appropriate because it begins at boundary point (0,0). 


(b) This could be the sequence of an interior point search. It starts at one interior 
solution, passes to another, and reaches the boundary only at the optimum. 


(c) This sequence starts in the interior but goes to the boundary before optimality at 
(1000,400). It could not result from an interior point algorithm. 


(d) This is the simplex algorithm extreme-point sequence of Figure 5.5. It never 
enters the interior. 
Interior in LP Standard Form 
In Section 5.1 we showed how every linear program can be placed in standard form: 
min (or max) ¢-x 
Sit. Ax=b (6.2) 


x>0 


Any main inequalities are converted to equalities by adding slack variables, and 
original variables are transformed until each is subject to a nonnegativity constraint. 
For example, we can place the Frannie model (6.1) in standard form by adding slack 
variable x3 in the main constraint to obtain 


max 90x1-+ 150x2 
s.t. 5X1 + x2 +43 = 3 (6.3) 

X1,%2,%3 2 0 
Chapter 5 placed LPs in form (6.2) to make it easier to perform simplex al- 
2orithm computations, but standard form is equally convenient for interior point 


search. With all inequalities reduced to nonnegativity constraints, it is easy to check 
-hether a given solution lies in the interior of the feasible region. 
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point if every comport 
_ Solution is strictly pos 


For example, point x of Figure 6.2 corresponds to x® = (1, 4, 2) in standard form 


> Ds bi 
(6.3). In accord with conditions , the vector is positive in every component. 


Consider the standard-form iieat program 


min 5x; — 2x3 + 8x4 

st. 2x1 + 3x. — x3 = 10 
6x, — 2x4 = 12 
X14, X2,X3,X4 > 0 


which does have strictly positive feasible solutions. Determine whether each of the 
following solutions corresponds to an interior point. 


(a) x = (8, 0, 6, 18) (6) x24) 131,6) (c)x@'='G; 3, 156) 
Analysis: 

(a) Solution x“ cannot be interior, because a component ad = 0 is not strictly 
positive. 


(b) Solution x” is positive in every component. Also, 
2x 3x0 — 4 — 24) + 3(1) — (1) = 10 
6x — 2x = 6(4) — 2(6) =12 
which establishes the point is feasible. Thus x is an interior point. 
(c) Solution x is also positive in every component. However, 
2x + 3x5 — x =2) +36) -— 1) = 144 10 
an Hine) = 6(3) — 2(6) =6412 


Thus the point is infeasible and so not interior. 


Projecting to Deal with Equality Constraints 


The ease of identifying interior points in standard form comes at a price. Many equal- 
ity constraints are added to the system Ax = b as main inequalities are converted, 
and equality constraints are active at every solution. Straightforward moves such as 
the objective vector directions of principle [6.1 | must now be modified to preserve 
the equalities. 

Chapter 3 (principle , Section 3.3) established that a direction Ax pre- 
serves equality constraints Ax = b if and only if the net effect on every constraint 
is zero. 


6.1 Searching through the Interior 


.4_| A move direction Ax is feasible for equ 
isties _. — 


How can we find a Ax direction satisfying conditions | 6.4 | that approximates 


as nearly as possible the direction d we would really like to follow? Interior point 
algorithms often use some form of projection. 


tem of equalities is a 
izing the total squared differ- 


Readers familiar with statistics may recognize this idea as the one used in least squares 
curve fitting. 

Figure 6.3 illustrates for Frannie’s firewood optimization form (6.3). The shaded 
triangle in this 3-dimensional plot is the set of feasible (x1, x2, x3). The search begins 
atx = (1, 5, D). 


x3 


26 


18’ 18’ 18 


optimal solution x* = (6, 0, 0) 
% 


FIGURE 6.3 Projection in the Frannie’s Firewood 
Example 


To improve the objective function as quickly as possible, we would like to move 
parallel to standard-form cost vector ec = (90,150, 0). Call the desired direction 
= ae = G, 3, 0) to keep it im the picture. 

A feasible direction Ax must keep the next point in the feasible plane by sat- 
siying , or 

5 Axy + Axp + Ax3 =0 (6.4) 
But we would like it to be as much like d as possible. Figure 6.3 shows that the best 
choice is d’s projection Ax = (#3, 4, — 38). It is the closest to d in the sense that 
(d, — Axi)4 + {da > Axe)? + (dy — Ars)? 


S minimized. It also satisfies feasibility condition (6.4), because 
2 (ia) + (Ga) + (-ig) = 0 
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Derivation of the projection computation that produced this move direction Ax 
is beyond the scope of this book. However, the formula for the required projection 
matrix P is well known: 


Here I denotes an identity matrix and A’ indicates the transpose of A, which is 
obtained by swapping its rows for its columns. (Primer 6 reviews some properties 
for those who may require it.) 

For our Frannie’s Firewood example, 


1 
) 
A= GAY, VATS tl AAT = 
1 
Thus (AA‘) a ¢, and 
i = Se! 
5 9. 9 
ail 
AT(AAT) A=| 1 /@G.1D=] 2 4 4 
1 ap) A 
9 9 9 
The needed projection matrix P then becomes 
P = (I— A‘(AA™)"'A) 
Te S442 2 
100 oa _ oF 5 9 9 
=[o10}-)3 3 $]=|-3 ¢ -3 
loghie eee it ee) Paras Py) 
gH oH 'g 9 9 9 
Applying rule 6. |ford = G, 3,0) gives 
Ax = Pd 
cS epee meee 2 3 4 
9 9 9 2 18 
= ee De pote 5 = 19 Cm 
3 9 9 9 Te 18 
2 4 5 
Ap ish qe 0 mie 


6.1 Searching through the Interior 


The transpose of a matrix M, 
columns of M. a Toros 


Ri is nonsymmetric, because R y' is sym 
len matrices can be mult -rimer 3, Section 5.1), the transpose 


sequence of multipli- a 


NIE ale 


SAMPLE EXERCISE 6.4: 


SERVE EQUALITY CONSTRAINTS 


Consider the standard- forth linear program 
max 5x; +7x2. +9x3 
s.t. ibal -lxy =-l 
+2x2 +1x%, = 5 
X45 XQ, X38 = 0 
a) Determine the direction d of most rapid objective function improvement at inte- 
rior point x = (2, 1, 3). 


b) Project that vector d to find the nearest direction Ax that preserves the main 
=guality constraints of the LP. 


c) Verify that your Ax satisfies all requirements for a feasible move direction at x. 
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Analysis: 


ng 


(a) Applying principle , the direction of steepest improvement is 


d = objective function vector = (5, 7, 9) 


(b) To compute the projection, we apply formula a lalerts) 


1 
il (Al = 
A= ; and. As = 0-2 
0 D 1 
ile eel 
Thus 
Vda rm; Brit eles 9° 9 
AA =( ey 5 ) with inverse (AA‘)™ = ( 1 oe 
8) 
Continuing yields < 
1 0 Sal 
ee) 1 _ 
AT(AAT)“1A = 0 2 9 9 0 1 
ila Or 2: 1 
=i! ad 9 9 
Ss en 
9 9 9 
= 2g 2 
a 519) 9 
ee 2 5) 
ns) 9 
and 
P = [I _ AT(AAT)“1A] 
De eed! Ae oe 4 
1 0 0 9 9 9 9 9 9 
Ws D Ag 2 cA) 2 1 
Hal lye A Oe eee act | | ag la. ae 
a er angst ee 
oe 5 9 9 9 9 
We conclude 
ae ne 4 14 
9 5 9 5 3 
zt EM D 1 2 
Ax = Pd = a 5 -% fi = —; 
4 Beri a 9 14 
9 9 9 


ee 4 
Adx=( 2 ‘) ae 


3 


With no inequalities (nonnegativities) active at x, this is all that is required for Ax to 


be feasible. 


6.1 Searching through the Interior 


Improvement with Projected Directions 


Projection principle | 6.6 | yields a feasible direction Ax at any standard-form interior 


point. But does the direction remain improving? For example, we know that the 
direction d = (3, 5, 0) used in Frannie’s Firewood computation (6.5) is improving 
because it parallels the maximize objective function vector ¢ = (90, 150, 0). Projec- 
tion would do little good if it achieved feasibility of Ax at the loss of this improving 
property. 

Fortunately, we can show that improvement is always preserved in projecting 
objective function vectors. 


For example, the Frannie’s Firewood direction Ax of (6.5) satisfies gradient improve- 
ment conditions | 3.21 | and of Section 3.3 because 


6 
c- Ax = (90, 150, 0) - (#3, #3, -72) 
= = > 0 


To see that this will always be true, we need to make a simple observation 
about projections. Computation finds the nearest vector Ax to direction d that 
satisfies AAx = 0. Thus if d already satisfies Ad = 0, the nearest such direction must 
be Ax = d itself. That is, re-projection of a Ax that has already been projected must 
leave Ax unchanged, or in symbols, 


projection(d) = Pd 
= PPd (6.6) 
= projection[projection(d)] 


Projection matrices are also known to be symmetric (P = P). Combining with 
property (6.6) applied to maximize objective function vector ¢, we can see that the 


corresponding Ax of principle has 

c- Ax = c'Pe 
c'PPc 
= c'P™Pc 
= (Pe)' (Pe) 
= Ax'Ax 
> 0 


This is exactly what is required for improvement, and a similar analysis holds for 
minimize models. 


SAMPLE EXERCISE 6. 


Return to the linear program of Sample Exercise 6.4 and its projection matrix P 
derived in part (b). Assuming that the objective function was changed to each of the 
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following, compute the Ax value obtained by projecting the corresponding steepest 
improvement direction of principle| 6.1}, and verify that the resulting move direction 
is improving. 


(a) max x1 — x2 +.%3 
(b) min 2x; + x3 
Analysis: 


(a) The corresponding steepest improvement direction is e = (1, —1, 1), so that the 
projected move direction would be 


eed 4 10 

9 9 9 1 9 

a ete 2 ey te? pe = =) 
Ax=Pc= 5 5 5 1 = 5 
ee wae & 1 10 

9 9 9 <0h 


Checking improvement yields 


ev Ax =, =1 1G =2, 5) = 3 > 0 


(b) For this minimize objective the steepest improvement direction is —e = (—2, 0, —1). 
Projecting gives 


sap is? 4 ee 

9 9 9 zu) 3 

= a2 ye 42, ae 2 
Ax= 9 9 9 eT 5} 
i ae! = _4 

9 9 9 ) 


Checking improvement yields 


6.2 SCALING WITH THE CURRENT SOLUTION 


We have already seen that it is important for interior point algorithms to avoid 
the boundary of the feasible set (until optimality). One of the tools used by virtu- 
ally all such procedures to keep in the “middle” of the feasible region is scaling— 
revising the units in which decision variables are expressed to place all solution 
components a comfortable number of units from the boundary. In this section 
we introduce the most common affine type of rescaling used in all algorithms of 
this chapter. 


Affine Scaling 


Affine scaling adopts the simplest possible strategy for keeping away from the bound- 
ary. The model is rescaled so that the transformed version of current solution x is 
equidistant from all inequality constraints. 

Figure 6.4 shows the idea for our Frannie’s Firewood example model (6.3). 
Part (a) depicts the original feasible space, with a current interior point solution 


6.2 Scaling with the Current Solution 


x = (3, 5, 1). The rescaled version in part (b) converts to new variables 


x1 x] 

A 

1 re ae 

y oe 
X2 x2 

y= Olas ls 
7) 2 
X3 x3 

¥3= Oa 
Xe 1 


After dividing by current (positive because interior) x-values in this way, the corre- 
sponding current solution y® = (1, 1, 1)—equidistant from all inequality (nonnega- 
tivity) constraints. 


optimal solution x* = (6, 0, 0) 


x2 


(a) Original space 


¥3 


? scaled feasible region 


y2 
(b) Affine scaled space 


FicurRE 6.4 Affine Scaling of Frannie’s Firewood Example 
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Diagonal Matrix Formalization of Affine Scaling 


Although it may seem excessive at this stage, it will prove helpful to formalize affine 
scaling’s simple notion of dividing components by the current solution in terms of 
diagonal matrices. In particular, we convert current solution vectors 


xO Se ay eae x) 
into the square matrix 
1 
Oo q 
i) x 
eae 0 : 
( i 
0 x; : =i “6 
» With = “Xess = x5 
, ie ee) ; 0 
0 OL ae 
0 1 
PG 
Affine scaling and unscaling can then be expressed in terms of multiplication by such 
matrices. 


6.8 | At current solution x® : 


olution x 


3 Se OO i 0 0 
xO=] 5 gives X,=| 0 $ 0 with X;'=| 9 2 0 
1 OO: 1 0 0 1 


Then under formula || 8 


; 0 0 3 1 
ys x = (ogo oy ied wal a. 
O01 1 1 


6.2 Scaling with the Current Solution 


arciguz6 i 3 
LoXy=| 0 5 10 1) | a hae 
001 1 1 


ate that the rete two pens one mared i a inbing programming ager algorithm 
are x = (12, 3,2)andx = (1, 4,7). Compute the affine scalings of x = (24, 12, 14) 
relative to each of these x, and verify that applying reverse scaling to the resulting 
y's recovers x. 


Analysis: For x the diagonal matrix of computation is 


1 
0) 00 eee 0 
been) Om ae | (RN, SR ae ae 
2 
0 0 Oak Gad 
Thus 
al 
coals 24 2 
Yoee x=] 0 5 0 Rls 
ja 14 a 
Reversing the scaling with formula [6.9 recovers 
12 0980 2 24 
LO voi (Oia 7 14 


After the search advances to x"), the scaling changes. Now the diagonal matrix 
of computation is 


elias duds 0 
X=] 0 4 0] with xjt=|] 0 7 0 
A fis 2 
Thus solution x = (24, 12, 14) scales to 
Dal \ age 24 
y=x, x=] 0 8 


14 


S 
oO sin 
is 
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Still, reversing the scaling with formula recovers 
1 0 0 24 24 
x=Xiy=] 0 4 0 Saleen el 
0 0 7 


Affine-Scaled Standard Form 


Affine scaling in effect changes the objective function and constraint coefficients of 
the LP standard form: 


min ormax ex 
s.t. Ax=b 


Substituting x = X,y from formula | 6.9} and collecting ¢- x = eX,y 4 ey and 


x: 


Our Frannie’s Firewood example has 
¢ = (90, 150, 0) 
A= G1), ab = 6) 
Thus the affine-scaled form corresponding to the x = (3, 7 1) of Figure 6.4 is 
max 270y; + 75y2 
s.t. 3y1 + 5Y2 ya =-3 


Y1; Y2, ¥3 2 0 
with 
8-10 
c® = eX, = (90,150,0)| 0 5 0 | = 270,75,0) 
O10 i 
3.G0.10 
A, =AX,= (Gd) 0 50 = 3.4,1) 
O= 70+ 


6.2 Scaling with the Current Solution 


_ After 7 moves in an improving search of the linear program 
min —3x; + 9x3 
st. —xy +3 =3 
x1 + 2x2 =4 
X1,X2,x%3 > 0 


) x”) = (2,1,5) has been reached. Derive the corresponding affine-scaled standard- 
form model. 


Analysis: Here 


—1. 01 
Ler) 


TO) 0) 
cX7 = (-3,0,9){ 0 1 0 


0 0.5 


c = (-3,0,9), A= ( 


® 


] 
di 
2 
S 
aS 
nN 
~~ 


Az 


ll 
> 
ea 
~ 
I] 
fo aS 
ys, le 
ae) 
Cor 
Ss 
—. 
Se: S&S 
oO | & 
noo 
eee 
ll 
SoS 
| 
N NY 
NY © 
Se 
Sat 


min —6y; + 45y3 

st. —2y, +5y3 =3 
2y1 + 2y2 
1, Y2, 3 = 0 


Projecting on Affine-Scaled Equality Constraints 


We have already seen in Section 6.1 (formula | 6.6|) that interior point algorithms 
often employ some form of projection to find directions that preserve the equality 
main constraints of standard form. We will soon see that most of those projections 


involve scaled problem coefficients e® and A, of affine-scaled standard form | 6.10 | ; 
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at current solution x = 
(2, 1,5), and compute the projection of direction d = (1,0, —1) onto correspond- 
ing the standard-form equality constraints in affine-scaled y-space. 


Analysis: We apply scaled projection formula . From Sample Exercise 6.7 we 


have 
heareos eX ot be 
OT Sea ee 
Thus 
8 4 
29 —4 j =] 216 216 
A,At = ( a 8 with (A7A7) = ( Ge? oe 
216) 21 
so 
25° 25) 10 
54 54 54 
P; = (I— A7(A7A7) 1A;) = -2 = -2 
10 10 4 


Thus the projected direction 


dso Sos Vin ze 
SA aS ne Sd 1 18 

a ti 2) 25 __ 10 2 = a 
Ay=Pd= 34 54 54 a 18 
ioe. dor = As rol 1 

serosa Sg 9 


Computational Effort in Interior Point Computations 


The distinction between revised projection formulas of and the unscaled for- 


mulas of may seem rather trivial. Haven’t we just changed from A, x, and P to 
Ay, y, and P;? 

In fact, the difference is much more profound and accounts for most of the 
computational effort in interior point algorithms. The Key insight is that the model 
constraint matrix A does not change as the algorithm proceeds. Thus its projec- 
tion matrix P does not change either. If algorithms required only projection onto 
Ax = b, some representation of P could be computed once and stored. Thereafter. 
each move would involve only choosing a desired direction and multiplying by the 
representation of P. 

Contrast now with the fact that algorithms actually have to project onto scaled 
constraints A,y = b (or something similar). Since A, 4 AX,, A, changes at every 
move with current solution x, it follows that new projection computations have 
to be performed at every move. Clever techniques of numerical linear algebra can 
do much to make the computation more efficient, but the fact remains that a very 
considerable effort has to be expended at each move of the search. 


6.3 Affine Scaling Search 


6.3 AFFINE SCALING SEARCH 


The affine problem scaling of Section 6.2 produces a new version of the model for 
which scaled current solution y® is = 1 in every component. Thus the transformed 
point is neatly equidistant from all boundary (nonnegativity) constraints. It is natural 
now to think of computing a search move in this simpler scaled solution space and 
then converting it back to the true decision variables x via unscaling formula 6.9]. 
In this section we develop an affine scaling form of interior point search for linear 
programs that adopts just such a strategy. 


Affine Scaling Move Directions 


Principle} 6.1 | tells us that the preferred move direction in the scaled problem follows 
objective function vector ¢®. Projection formula computes the closest Ay 
satisfying feasible direction requirement A,;Ay = 0. Combining gives the direction 
Ay =P,c” (6.7) 
for a maximize problem, and 
Ay = —P,c® (6.8) 


for a minimize problem. 
To complete a move direction we have only to translate back into the original 


variables via inverse scaling formula F 


(+ to maximize and - —to minimize) where cf) 


Xx, aad P, are as in definitions 


We can illustrate with the initial solution x = (1, 4 , 2) to our Frannie’s Fire- 
wood example of Figure 6.4. Scaled standard-form definition [6.10 makes 


17HL 0 
ce = Xo = (90, 150,0)] 0 4 0 | =(90,75,0) 
OO} 2 
and 
1 0256 
Ay =AX)= (1, | 0 + 0 |—C,!,2 


o 
Oo MPR 
i) 
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(6.9) 


Return to the linear program of Sample Exercises 6.7 and 6.8 at current solution 
x = (2,1,5). Compute the next affine scaling move direction. 


Analysis: From Sample Exercise 6.7, 


ce = (-6,0,45) and A,= ( 


and from Sample Exercise 6.8, 


20a 
a eS (0) 


Continuing with formula for a minimizing problem gives 


2 
Ax® =—]| 0 
0 


£05 
34 
25 
34 


east) 
54 
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Feasibility and Improvement of Affine Scaling Directions 


Alert readers will notice that our derivation of direction is guided by rules to 
produce an improving and feasible move in scaled y-space. But we are using the 
direction in the original x-space. Will the familiar improving and feasible direction 
properties of improving search be preserved in that original problem setting? 
Fortunately, the answer is yes. 


6.3 Affine Scaling Search 


For feasibility, the AAx = 0 required by condition is the same as AX,Ay = 0, 
which must be true when projected on A,;Ay = AX,Ay. The argument for improve- 


ment is much the same as the one given for unscaled projection [67 in Section 
6.1. 
We may illustrate {with Frannie’s Firewood direction 


os (802, 324, —734) 
of expression (6.9). Since we are in the interior, feasiblity requires only : Ax, + 
Ax) + Axz = 0 (condition | 6.4|). Checking, we find that 
2 (803) + (3243) + (-733) = 
Similarly, improvement for this maximize problem requires that ¢ - Ax > 0. Here 


(90, 150, 0) « (808, 321, 731) ~ 12212 > 0 


Affine Scaling Step Size 


With directions of construction | 6.13 |in hand, the next question involves how far to 
move. That is, we need to choose an appropriate improving search step size i. 

As usual, there may be no limit on progress in direction Ax. For linear programs 
in standard form, the only inactive constraints at interior point solution x > 0 are 
the nonnegativities. If Ax decreases no component of the solution, we can improve 
forever without encountering a nonnegativity constraint. 


6.15 | A linear program in standard form 1 is unbounded if affine cab’ con- 
struction | 6.13 [6.13] ever epmocuces a Graces > 0. 


When affine scaling produces a move direction with some negative components, 
we must limit the step so that 


xD —~ x94) Ax >0 


point algorithms avoid the boundary uatila an optimal palate 3 is obtained. That is, 
we should move all the way until a nonnegativity constraint is active only if such a 
move will produce an optimal solution. 

There are many step size rules that fulfill both of these requirements. The 
ye we will adopt is best understood in affine-scaled space over y-variables. Scaling 
makes the current solution there a vector of 1’s. For example, Figure 6.5 displays 
current scaled solution y© = (1, 1, 1). The highlighted unit sphere around y shows 
sow a move of length 1 in any direction maintains all nonnegativity constraints. We 
can implement affine scaling by stepping to the limits of such a sphere. 


6.16 At the current feasible solu 
form, if the Ax computed from 


ear program in standard 
1 “nts, affine 
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FIGURE 6.5 Feasible Unit Sphere in the Frannie Firewood 
Example 


Figure 6.5 shows this move to the limit of the sphere along the direction com- 
puted at equation (6.9). The scaled direction is 


100 802 802 
Ay=X)'Ax| 9 2 0 324 |=] 652 
00 3 -73} « —365 


with length (norm) 


\Ayll = ,/(605)? + 58)? + (362)? ~ 110.5 


Thus the computation | 6.16 | step of 


1 1 
A= —— = —— (0.00905 - 6.10 
[idyll 1105 em 


brings us exactly to the boundary of the unit sphere. 
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SAMPLE EXERCISE 6.10: COMPUTING AFFINE SCALING STEP SIZE 


_ Determine the appropriate step size to apply to the move direction Ax = (-112,5 
—11 4) derived in Sample Exercise 6.9 at point x = (2, 1,5). 


Analysis: Applying principle yields 


- Diy Os VE -114 —53 
Ay = X7Ax— leo) 1-0 53 |=] 53 
ON OE ene 192 


This Ay has 


||Ay|| = Se + (53)? + (23)? = 8.165 
Thus the appropriate step is 


1 


Termination in Affine Scaling Search 


Applying the step computed in expression (6.10) advances our search of the Frannie’s 
firewood example to point 


x) = x + Ax 
= (0.5, 1, 1) + 0.00905(80.83, 32.92, —73.33) (6.11) 
= (1.73, 0.80, 1.34) 


Notice that the new point remains positive in all components and thus in the interior 
>i the feasible region. 

With step rule constructed to keep updated (scaled) points within the 
anit sphere of feasible y, the new solution will always be interior unless the search 
moves to one of the points where that sphere touches a nonnegativity constraint. 
Such moves are rare, but if they occur, the new solution can be shown to be optimal. 

Much more typically, an affine scaling search will remain in the interior, stepping 
ever closer to an optimal solution on the boundary. Then a stopping rule is required 
to terminate computation when a solution becomes sufficiently close to an optimum. 

But how do we know that we are near an optimal solution? Crude schemes 
simply stop when the solution value is no longer changing very much (as with Algo- 
othm 6A). More precise rules derive bounds on the optimal solution value at each 
step so that we can know how much room remains for improvement. We look at the 
atter briefly in Section 6.5. 


Affine Scaling Search of the Frannie’s Firewood Example 


‘Jgorithm 6A collects all the insights of this section in an affine scaling algorithm for 
near programs in standard form. Table 6.1 and Figure 6.6 detail its application to 
or Frannie’s firewood example. 
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Step 0: Initialization. Choose any starting feasible interior point solution, 
x > 0, and set solution index ¢ < 0. 

Step 1: Optimality. If any component of x is 0, or if recent algorithm steps 
have made no significant change in the solution value, stop. Current point 
x® js either optimal in the given LP or very nearly so. 

Step 2: Move Direction. Construct the next move direction by projecting 
in affine-scaled space as 


Ax]) © +X,Pc0 


(+ to maximize, — to minimize) where X,, P;, and e® are the scaled values 
of Section 6.2. 

Step 3: Step Size. If there is no limit on feasible moves in direction Ax“+) 
(all components are nonnegative), stop; the given model is unbounded. 
Otherwise, construct step size 


1 
A xT || 
Step 4: Advance. Compute the new solution 
KD 2 xO 4 2A CHD 


Then advance t <— t+ 1, and return to Step 1. 


TABLE 6.1 Affine Scaling Search of the Frannie’s 
Firewood Example 


nal x2 X3 
max c 90 150 0 b 
A 0.5 1 1 3 
x) 1.00 0.50 2.00 e+ x = 165.00 
Ax) 80.83 32.92 -73.33 A = 0.00905 
x) 1.73 0.80 1.34 ¢- x) = 275.51 
Ax?) 160.94 4925 129.72 A = 0.00676 
x2) 2.82 1.13 0.46 +x) = 423.40 
Ax®) 87.26 = -10.29 -33.34 A = 0.0126 
x@) 3.92 1.00 0.04 e¢-x@) = 502.84 
Ax“ 4813 -23.79 -0.27 A = 0.0362 
x(4) 5.66 0.14 0.03 e+ x) = 530.45 
Ax®) 1.49 -0.58 -0.16 A = 0.1472 
x) 5.88 0.05 0.01 e+ x = 537.25 
Ax) 0.19 -0.09 -0.01 A = 0.5066 
x() 5.98 0.01 0.003 e+ x) = 539,22 
Ax 0.008  —0.003 -0.001 A = 1.7689 
x 5.99 0.005 +0.000 +x = 539.79 
Ax) 0.001  -0.001 —0.000 A = 6.3305 
x) 6.00 0.001 +0.000 ¢-x@) = 539.95 
Ax 40.000 0.000 -0.000 A = 23.854 
x) 6.09 +0.000 +0.000 e+ x) = 539.99 
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\ feasible region 


x3 


> xX 
x4) 


optimal solution x* = (6, 0, 0) 


Xx 


FIGURE 6.6 Affine Scaling Search of the Frannie's Firewood Example 


Notice that the search makes very rapid progress while it is near the “middle” 
of the feasible region. Later iterations approach the boundary but never quite reach 


«=. We terminate after the ninth iteration changes the objective function by less 
than 0.1. 


6,4 LOG BARRIER METHODS FOR INTERIOR POINT SEARCH 


Affine scaling is only one way to keep algorithms away from the boundary. In this 
section we develop a log barrier alternative. 


Barrier Objective Functions 


Remembering that the boundary of a standard-form linear program is defined by 
sonnegativity constraints, log barrier methods exploit the fact that In(xj) > —oo as 
« — 0. A modified barrier objective function includes In(x;) terms to keep the x; 
sway from the boundary. 


where p, > 0 is a sp 
inimize objecti 
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We can illustrate with our familiar Frannie’s firewood standard form 
max 90x, + 150x> 
s.t. 5X1 +%2+%x3=3 
X1,X2,x3 = 0 
Adding a barrier term produces the modified model 
max 90x; + 150x2 + w [In(@x1) + In(x2) + In(x3)] 
s.t. 5x1 +%2+%3=3 (6.12) 
X1,X2,%3 > 0 


Suppose that we (arbitrarily) choose multiplier , = 64. Then the objective at 
feasible point x = (2, 1, 1), which is far from the boundary, evaluates to 


90(2) + 150(1) + O(1) + 64 [In(2) + In(1) + In(1)] © 374.36 


The log-barrier terms do have an influence, because the true objective value at this 
x is 90(2) + 150(1) = 330. Still, the difference is modest. 

Compare with near-boundary feasible point x = (0.010, 2.99, 0.005). There the 
barrier objective function evaluates as 


90(0.010) + 150(2.99) + 0(0.005) + 64 [In(0.010) + In(2.99) + In(0.005)] 
= 449.4 + 64(—4.605 + 1.095 — 5.298) 
x —114.31 


The corresponding true objective value is 90(0.010) + 150(2.99) = 449.4. We see 
that the negative logarithms of x; near 0.0 have severely penalized this solution in 
the modified maximize objective. 

It is in this penalization sense that modified objective functions of construction 
erect a “barrier” to near-boundary solutions. As components x; approach their 
boundary value of 0.0, penalties become larger and larger, thus guaranteeing that no 
improving search will choose to approach the boundary too closely. 


JECTIVES 


Linear program 
min 5x; + 3x2 
s.t. Xy+%.>1 
O<x, <2 
O<x2 <2 


has feasible region as displayed in the following figure: 


6.4 Log Barrier Methods for Interior Point Search 


2) 


optimum 
= (0,1) 


(a) Place the model in standard form. 


(b) Show the modified objective function obtained when a logarithmic barrier is 
introduced to discourage approaching the boundary. 


(c) Choose a solution near the middle of the feasible region, and compare the true 
and modified objective functions at that point using w = 10. 


(d) Choose an interior point solution near the boundary of the feasible region, and 
compare the true and modified objective functions at that point using the same 
p = 10. 


Analysis: 
(a) Introducing slack variables x3, x4 and xs, the standard-form model is 
min 5x; + 3x2 
st. xy +X. —2x3=1 
x1 +Xx4=2 
x2 +x5=2 
ites SAW) 


b) Following construction , the barrier objective with multiplier jz is 


min 5x, + 3x2 — w [In(@x1) + In(v2) + In(x3) + In(v4) + In(xs]) 


From the figure it is clear that x; = x2. = 3 is near the middle of the feasible 


c) 
region. There the true objective function value is 5 (3) + 3(3) = 10. Corresponding 
values for slack variables are x3 = 3 + 3 -1l= : and x4 =x5 =2— 3 = 3. Thus the 
log barrier objective with . = 10 evaluates 

5(3) + 3(3) — 10 [In(3) + In) + In) + In) + In] © 7.237 
Modified cost is a bit lower that the true objective value. 


d) One point near the boundary is x; = 1.999, x. = 0.001. There the true ob- 
‘ective function value is 5(1.999) + 3(0.001) = 9.998. Corresponding values for 
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slack variables are x3 = 1.999 + 0.001 — 1 = 1.000, x4 = 2 — 1.999 = 0.001, and 
x5 = 2 —0.001 = 1.999. Thus the log barrier objective with 4. = 10 evaluates 


5(1.999) + 3(0.001) — 10 [In(1.999) + In(0.001) + In(1.000) 
+ In(1.999) + In(0.001)] + 134.3 


This near-boundary point is penalized severely. 


Problems with Gradient Directions 


To make any progress on our underlying LP with barrier methods, we must find 
improving feasible directions for the now nonlinear standard form 


max ormin f(x) 4 os [ejx; + wIn(;)] 
j (6.13) 
s.t. Ax=b 
Nonnegativity constraints can be ignored because the barrier objective will keep us 
away from the boundary. 

Although the gradient-based directions introduced in Section 3.3 have served 
us well so far, they usually perform poorly with nonlinear objective functions (see 
Section 13.5 for details). In essence they treat the objective as if it were approximated 
as 


a 
fK® +2Ax) © fx) 4a) ae Ax; (6.14) 
j J 


where x" is the current solution and Ax a proposed move direction. This approxima- 
tion, known as the first-order Taylor series (see also Section 13.3), takes the function 
to be roughly its current value plus the net effect of partial derivative slopes times 
move components. Choosing gradient-based direction Ax = +Vf(x™) produces the 
most rapid change in the approximation per unit A. 

With a linear objective function, partial derivatives are constant and approx- 
imation (6.14) is exact. But with nonlinear cases, the slope information in partial 
derivatives can decay rapidly as we move away from point x. The result is that 
a direction based on rapid progress for the simple (6.14) approximation may prove 
very ineffective for the real nonlinear objective. 


Newton Steps for Barrier Search 


For those who remember their calculus (refer to Primers 2 and 7 if needed), it is 
natural to think of enhancing approximation (6.14) with second partial derivatives. 
The corresponding second-order Taylor form is 


fe +.Ax) ® f+," =: Ax; 
‘patel (6.15 


New terms account for changes in the rates of change df/dx; as we move away 
from x. 
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The partial derivatives needed in this new approximation have a particularly 


a 
Ox; Xj 
aie 6.16 
af =a if j=k 16) 
= j 
OX} OXK 0 otherwise 


Thus fixing 2 = 1 for the moment and taking second-order approximation as exact, 
it makes sense to choose a move direction Ax at x that solves 


maxormin f(x + Ax) ~ Din Gt a =e ue 


2 
j x eee ) 
s.t. AAx = 0 


That is we choose the Ax that most improves our barrier objective function—as ap- 

proximated by Taylor expression (6.15)—subject to the familiar requirements (con- 

ditions [6.4]) that the direction preserve all equality constraints of standard from 
6.13). 

Using Lagrange multiplier methods beyond the scope of this section (but devel- 

oped in Section 14.3), it can be shown that the Ax which solves the direction-finding 


problem (6.17) is remarkably like the affine-scaled steps of Section 6.3. It takes the 
form 


2 
is (6.17) 


Oty 
1 ; 
Ax =+-XP, (6.18) 
m 
Ch ip 


where X,, P;, and e are the scaled problem data of and in Section 6.2. 


The only difference from affine scaling direction is the inclusion of barrier 
multiplier jz in forming the scaled direction to project. 
This move is often called a Newton step because it is based on the same second- 
rder Taylor approximation (6.15) that gives rise to the famous Newton method for 
=guation solving and unconstrained optimization (see Section 13.6 for a full devel- 
»pment). Allowing a step size 4 to be applied, which means that we may drop the 
cading constant 1/j, yields the directions to be employed in our barrier form of 
nterior point LP search. 


6.18 | ANewtonstep barrier algori as reached feasible solution x > 0 
with barrier multiplier 1 > 0 moves next in direction 


=H 
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To illustrate construction |‘ , we return to our Frannie’s Firewood example 
atx = (1, $,2) with » = 16. Scaling of Section 6.2 gives 
iy 0 0 
¢ = eX = (90, 150,0)] 0 4 0 | = 0,75, 0) 
Ore? 
and 
Ay = AXy = (4, 1,1) 
Thus 
Tale tet ny ans 2 
18 18 9 
Pe (I 2 A}(AAJ)"1Ao) =i bodice stioliue 
BRIS 1 
9 9 9 
Now projecting to find the next move direction, 
oo 1 
Ax” = XoPo 
cy +h 
17 1 2 
1. 0.48 i. yinis 7B 90 + 16 
1 1 17 2 
= | 9 3 —% i —3 75 + 16 
0 0 He ee 1 0+ 16 
9 9 9 


II 


(914, 384, —84) 


Previous Samle Exercises 6.7 amd 6.8 conaneved the rode 


min —3x, + 9x3 

st. —x1 + x3 = 5) 
x1 + 2x2 =4 
X1,X2,x%3 > 0 


at current solution x‘? = (2, 1,5). Compute the next Newton step barrier search 
direction assuming a barrier multiplier 4. = 120. 
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Analysis; From the earlier sample exercises we know that 
ec) = cX, = (—6;0,45) 
at the specified x, 


Ore Git 5 
Ay = AX = ( baa) 


54 

eld 

54 

da 

54 

a 

2 _25 10 331 
2 0 0 54 54 54 —6— 120 3 

wie 25 25 _ 
=-|0 1 0 -— 3 -3 0-120 }=| —163 
0-05 ies a6 4 45 — 120 1 
S454 OS 333 


Newton Step Barrier Search Step Sizes 


As usual, our next concern is how big a step to take in barrier search direction ; 
Any such rule must first assure that 
xD — x9 4 Ax) 5 9 


© that the new point will also be in the interior. 
This is easily accomplished by familiar minimum ratio computations. To remain 
terior, we keep A less than or equal to say 90% of the maximum feasible step size, 
that is, 
h < 0.9Amax (6.19) 
where 


xo 
ee nin | hr : on < | 
= Ax; 


With barrier objective nonlinear, there is also the possibility that im- 
-rovement stops before we approach Amax. A direction that improves near current 
= may begin to degrade the barrier objective function value after a larger step. 

Figure 6.7 illustrates both cases. At Fannie’s firewood x = (1, 5. 2), direction 


Ax = (91.5, 38.25, —84) decreases only x3. Thus expression (6.19) yields 
DZ 
Imax = 0.9 — } = 0.0214 
Oe = 0 9(5;) 0 
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Part (a) of the figure shows the barrier function improves throughout the range 
0<)’ < 0.0214. 


500 
480 a 
o 
i) 
s 460 
f= 3 
io) maximum at 
3 a 0.0214 
& 420- 
5 
& 4007 
Ss 
Oo 
380 |- 
150 | | | | 360 | | l | | | | 
0 0.005 0.01 0.015 0.02 0 02 04 06 O08 1 12) ea 
step size A step size X 
(a) Step from x© (b) Step from x 


FIGURE 6.7 Frannie’s Firewood Barrier Function Changes with Step Size 


Contrast with the more typical step at x = (5.205, 0.336, 0.062). The corre- 
sponding Ax“*) = (—3.100, 1.273, 0.278). Only x; decreases and the minimum ratio 
expression (6.19) gives 


1G) 
3 5.205 
0.9| —1+_ } =0.9( —— ) =1.511 
(=r) (sim) 

However, Figure 6.7(b) shows that the barrier function reaches a maximum long 
before this limit is encountered. 

We need an enhancement of the basic minimum ratio step size rule to account 
for this possibility of initial improvement followed by worsening of the barrier ob- 


jective. Fortunately, one is readily available in the Newton step computations of 
expression (6.18). That best move for the second-order Taylor approximation con- 


sisted of 1/j: times the barrier search direction of construction : 
Combining this idea with a minimum ratio gives a step size that both keeps to 
the interior and approximately optimizes the barrier objective: 
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seancu STEP zz 


(8) 


| (a) Determine the maximum step that can be applied to the direction Ax” computed 


_ there before the boundary is reached. 


(b) Sketch the barrier objective function for that model as a function of the step size 
i. applied to direction Ax®. 


(c) Compute the step size that would be applied by a Newton step barrier search 
algorithm. 


Analysis: 
(a) In Sample Exercise 6.12 we computed the next move direction, 
Ax® = (334, —163, 334) 
Applying the minimum ratio computation (6.19), the boundary would be reached at 


x? 
Amax = min eed 3 ar-ol 
— Ax; 


min Lag 
163 


= 0.060 
We should stop before, say, 90% or A = 0.9(0.060) = 0.054 to stay interior. 


b) The barrier objective function for this exercise is 
min —3x; + 9x3 — w [In(x) + In(v2) + In(vs)] 


Plotting as a function of step size gives the following: 


—100 


-120b 
-140 + 


-160/- 
-180/,- 
-200 


barrier function value 


-220- 
—240 


| ! l l ! fies af 
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 


step size X 


We seek the A that yields a minimum value without leaving the interior. 


283 


284 


Chapter Six e Interior Point Methods for Linear Programming 


(c) Applying rule , Newton step barrier search would use step size 


1 
Ree min | 7. 0.9imex| = min{7-0.054} = 5 = 0.00833 
LL 


Impact of the Barrier Multiplier 

Having worked out all the details of a barrier search for any fixed yw in barrier forms 
[6.17], it is time to consider how to manage that barrier multiplier. Parameter yw 
controls how much weight is assigned to keeping a search away from the boundary. 
For example, Table 6.2 details the effect for our Frannie’s Firewood barrier form 
(6.12). The optimal barrier problem values of x1, x2, and x3 are shown for a range 
of barrier multiplier values. For example, high weight 1 = 2! = 65,536 on loga- 
rithmic barriers makes the optimal x in model (6.12) approximately (2, 1, 1). As the 
multiplier decreases to = 2° = + the optimal x in (6.12) approaches the true 
optimum (without barriers) of x* = (6, 0, 0). 


TABLE 6.2. Impact of 1 on the Frannie’s Firewood Example 
Se eee 
Le xy X2 X3 vy X2 x3 


2'6 — 65,536 2.002 1.001 0.998 | 25 =32 4.250 0.711 0.164 
255 = 32,768 2.004 1.001 0.997 | 24 =16 4.951 0.439 0.086 
244 — 16,384 2.009 1.002 0.993 | 23 5.426 0.243 0.044 
23 = 8192 2.017 1.005 0.987 | 22 5.702 0.127 0.022 


= 8 

= 4 
2! = 4,096 2.034 1.009 0.974 | 2! = 2 5847 0.065 0.011 
24 = 2,048 2.068 1.018 0.948 | 29 = 1 5.923 0.033 0.006 
20 = 1,024 2.134 1.035 0.898 | 27= 4 5.961 0.017 0,003 
2 = 512 2.261 1.060 0.809 | 22= } 5.980 0.009 0,001 
2 = 256 2.494 1.088 0.665 | 23- 1 5990 0.004 0.001 
27 = 128 2.889 1.079 0.477] 24= 4 5.995 0.002 0.000 
2 = 64 3.489 0.960 0.295 | 25= 4 5.997 0.001 0.000 


as High values of barrier weight 1: > 0 severely a interior _ near 
_ the boundary. Low values encourage the searc 1 to approach the boundary. _ 


Barrier Algorithm Multiplier Strategy 


The power of multiplier to control how close barrier optimal solutions come to the 
boundary suggests a strategy for using barrier methods to solve the underlying LP. 


Initial moves seek good solutions far from the boundary, with later ones comin g ever 
closer to an optimum in the LP as barrier multiplier » > 0. 


Newton Step Barrier Algorithm 


Algorithm 6B collects all the ideas developed so far in a Newton step barrier algo- 
rithm for linear programming. The search begins with an interior (feasible) point 
x and a relatively large barrier multiplier 2. For each #4, an inner loop seeks to 
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approximately optimize the corresponding log barrier problem. That is, we take one 
or more steps to approach the barrier model optimum for that jy. 


Step 0: Initialization. Choose any starting feasible interior point solution, 
x > 0, and a relatively large initial barrier multiplier jz. Also set solution 
index t — 0. 

Step 1: Move Direction. Construct the next move direction by projecting 
in affine-scaled space as 


ee SE LL 
Ax) ~— +X,P, 


C 
P+ 


(+ to maximize and — to minimize) where X,, P;, and e are the scaled 
problem data of Section 6.2. 
Step 2: Step Size. Choose a step size 


A <—min {<. 0.9m] 
| 


where 


x” 
ei, hea i ee er Ax FED ce 0 
ay? ey J 


Step 3: Advance. Compute the new solution 
xltbD — x oe rAxOtD 


Step 4: Inner Loop. |f progress indicates that x) is far from optimal for 
the current jz, increment tf < ¢ +1, and return to Step 1. 

Step 5: Outer Loop. If barrier multiplier 4 is near zero, stop. The current 

point x‘+) is either optimal in the given LP or very nearly so. Otherwise, 

reduce the barrier multiplier z, increment ¢ <— t+ 1, and return to Step 1. 


Once slow progress indicates that we are near enough to the optimal value in 
the barrier model, an outer loop reduces jz and repeats the inner loop. The process 
-erminates when yp nears zero, indicating that the current solution is very close to an 

ptimum in the LP. 

As with affine scaling Algorithm 6A, many details of stopping and convergence 
sre left vague in the statement of Algorithm 6B because they involve mathematical 

ssues beyond the scope of this chapter. However, in Section 6.5 we provide further 
ssight regarding how such issues are handled in commercial-quality algorithms. 


Newton Barrier Solution of Frannie’s Firewood Example 


Table 6.3 illustrates Algorithm 6B on our Frannie’s firewood example. The ta- 
“le shows both the true objective function value at every step and the modified 
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objective value, including log-barrier terms (in parentheses). Figure 6.8 tracks progress 
graphically. 


TABLE 6.3 Newton Barrier Search of Frannie’s 
Firewood Example 


xy X5 X3 
max c 90 150 0 b 
A 0.5 1 1 3 


x 1,000 0.500 2.000 obj = 165.00 (165,00) 


Ax 91.500 38.250 -84.000 = 16.0, = 0.0214 
x) 2.961 1.320 0.200 obj = 464.41 (460.46) 


Ax? 59,996 -26.113  -3.885 1 = 16.0, A= 0.0455 
x?) 5,689 0.132 0.023 obj = 531.84 (467.12) 


Ax®) 3.519 1.487 0.272 p= 16.0, A = 0.0625 
x@) 5.470 0.225 0.040 obj = 526.00 (477.93) 

Ax) = 4.227 1.771 0.343 = = 16.0, A = 0.0625 
x) 5,205 0.336 0.062 obj = 518.82 (483.19) 


Ax®) 3.100 1.273 0.278 = 16.0, A = 0.0625 
x) 5.012 0.415 0.079 obj = 513.31 (484.44) 


Ax — -0.917 0.359 0.099 = 16.0, A= 0.0625 
x) 4.954 0.438 0.085 obj = 511.52 (484.51) 


Ax 6.804 -2.756 -~0.646 p= 8.0,A=0.1188 
x) 5.762 0.110 0.009 obj = 535.16 (493.42) 


Ax®) 1.076 0.483 0.055 w=8.0,A= 0.1250 
x®) 5.628 0.171 0.015 obj = 532.11 (498.40) 

Ax® 0.426 ~~ -0.232 0.019 w= 4.0,A = 0.2500 
x) 5,734 0.113 0.020 obj = 533.01 (515.63) 


Ax) 0.118 0.052 0.007 y= 4.0,4 = 0.2500 
x10) 5705 0.126 0.022 obj = 532.29 (515.67) 


Ax) 09.552, -0.233. 0.043. pw = 2.0, = 0.4614 
xf) 5.959 0.018 0.002 obj = 539.06 (522.36) 


Ax?) 0.059 0.026 0.004 pu = 2.0,A = 0.5000 
x(12 5.930 0.031 0.004 obj = 538.35 (523.91) 


Ax@3) — -9.006 0.002 0.001 gu =1.0,A= 1.0000 
x2) 5.904 0.033 0.005 obj = 538.10 (531.18) 


Ax“'4) 0.001 0.000 0.000 ju =1.0,A= 1.0000 
x4) 5,923 0.033 0.006 obj = 538.02 (531.18) 
Ax“) 0.038 = -0.016 += -0.003 w= 0.50,A=1.8 
x5) 5,992 0.003 0.001 obj = 539.80 (534.10) 
Ax(!6) 9,003 0.001 0.000 p= 0.50, = 2.0 
x16) 5.986 (1.006 0.001 obj = 539.64 (534.53) 


Initial point x is interior and satisfies the only equality constraint 5X1 +XxX2 + 
x3 = 3. We begin with barrier multiplier » = 16. After six moves, the search arrives 
at point x, which very nearly optimizes barrier objective function 


max 90x; + 150x2 + 0x3 + 16 [In(Q1) + In(@2) + In(x3)] 


Now we begin to reduce yz. In Table 6.3 each reduction updates w <— $ ue. 
although /« would normally not be reduced so rapidly. Two steps follow each change 
to bring the search near the best point for each new yz. 


6.5 Dual and Primal-Dual Extensions 


\ feasible region 


optimal solution x* = (6, 0, 0) —~ 


FIGURE 6.8 Barrier Search of Frannie’s Firewood Example 


Algorithm 6B terminates when yz approaches zero and no further progress is 
»cing made. Here we stop after ~ = $ with x" = (5.986, 0.006, 0.001), which is 
ery near the true LP optimum x* = (6, 0, 0). 


6.5 DUAL AND PRIMAL-DUAL EXTENSIONS 


in the interest of reducing the background needed to understand the notions behind 
sterior point search, the development of affine scaling and barrier algorithms in 
Sections 6.2 to 6.4 omitted many details of stopping and convergence rules needed 
= real implementations. To help complete the picture, in this section we sketch 
sriefly some of the missing elements. A full understanding requires familiarity with 
‘se development of LP duality in Chapter 7; the presentation of Newton’s method in 
Section 13.6, the treatment of Karush-Kuhn—Tucker optimality conditions in Section 
+4. and more. Still, readers who have absorbed the rest of this chapter should be 
»>e to follow the main concepts without further preparation. 


Duals of Standard-Form Linear Programs 


‘ow and for the remainder of this section, assume that we are dealing with the 
~ nimizing standard-form linear program 


min c:x 
st. Ax=b (6.20) 
x>0 


<so assume that the rows of A are linearly independent and that the model is feasible 
2d bounded, so that it has a finite optimal solution. 

In Chapter 7 we explain in detail how a closely related dual linear program 
san be posed over the same constants A, b, and ¢ as this given LP, now called the 
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primal. Specifically, the dual assigns variables v to the rows of the main constraints. 
Including slack variables r > 0, the dual’s standard form becomes 


max b-v 
st. vA+r=c (6.21) 
vURS, r>0 


This dual is a maximize problem because the primal minimizes. There is one dual 
constraint for each column or primal variable x;. Right-hand sides b of the prima! 
become objective function coefficients in the dual, and vice versa. 
To illustrate, return to the minimizing standard-form LP used in Sample Exer- 

cises 6.7 to 6.13. Its primal is 

min —3x,; +0x2. +9x3 

st. —1x, +0x. +1%3 

+1x, 42x, +0x3 =4 


X41, X2,%3 > 0 


(6.22) 


Assigning dual variables v; and v2 to first and second main constraints, respectively. 
the corresponding dual is 


max +3v, +42 


s.t. —ly, 4+1y. +n" =-3 
+0vy +2v2 +12 = () (6.23 
+1, +0v2 +r3= 9 
V1, V2 URS 


11,172,173 = 0 


A primal LP and its duals are related in far more ways than a surface rows- 
versus-columns symmetry. Most significantly, if either has a finite optimal solution. 
they both do, and the optimal solution values are the same. For example, primal (6.22 
has optimal solution x* = (0, 2,3) with value 27, and its dual (6.23) has optimum 

= (9,0), r* = (6, 0, 0) with the same optimal value. 

For our present purposes the most important primal—dual relationship connects 
primal solutions x in (6.20) and dual solutions (y, r) in (6.21) that are not yet known 
to be optimal. 


= gap 


Thatis, any feasible primal solution gives an upper bound on the mutual optima’ 
value, any feasible dual solution provides a lower bound, and the expression x - F 
measures the difference. For example, solution x = (2, 1, 5) is feasible in primal (6.22 


6.5 Dual and Primal-Dual Extensions 


with value 39, and (v, r) = (4, —1, 2, 2, 5) is feasible in dual (6.22) with value 8. Their 
mutual optimal value 29 falls between, and the gapisx - r = (2,1,5) + (2,2,5) = 31. 


Using Duals to Bound Affine Scaling Progress 


Algorithm 6A of Section 6.3 can be called a primal search because it moves through 
a sequence of feasible solutions to primal LP (6.20). Still, a corresponding dual 
solution can be constructed at each iteration. If that dual solution happens to be 
feasible in (6.21), we can apply relationship to bound how far our current x 
is from optimal. 

Recall (formula [6.13 | ) that Algorithm 6A employs (minimize problem) move 
cirections 


Ax = —X,P,c? = —X; (I — A} (A;A?)~'A,) c 


where A, = AX,, ce = eX,, and X, is the diagonal matrix of current solution 
x. In the process of carrying out this directional computation, it is straightforward 
to extract an estimated dual that will measure our progress toward optimality if it 
sappens to be feasible. 


6.23 | Asa by-product of direction forming in affine scaling Algorithm 6A for 
standard- os oo LP (6.20), estimated dual variable values can be 


0 = (AANA? 
oo rr? =e-vA : _ - 
Then ifr > 0, the difference between current solution value c - x andanLP 
optimum is bounded by xO or”, If this does not lead to a stop, the next move — 


For our example at x = (2, 1,5), we have already computed in Sample Exer- 
sses 6.7 and 6.8 that 


8 4 —6 
Ce ee 
216 © 216 45 
‘ese lead to estimated duals 
v = (A7A7A2) | Ae” = (88,22) 
r =ce-—wA = (23, -53, 3) 
With > =-—5 3 < 0, we do not obtain a bound, but the next move direction can still 


‘= computed: 
Ax® = — (X7)’r® = (-115, 53, -11}) 
= the other hand, feasible (and indeed optimal) dual estimate (v, r) = (9, 0, 6, 0, 
results from computation if we use x = (0, 2,3). Thus, as we move very 
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close to that optimal primal solution, bounds from the estimated duals would alert 
us that further progress is impossible. 


Using Duals to Bound Newton Barrier Search Progress 
Newton step barrier Algorithm 6B is also a primal search, and the same sort of dual 
bounding strategy can be employed to know how close it has come to optimality. 


The only differences relate to the fact that direction (for minimize problems) 
projects a vector adjusted for the barrier multiplier pu: 


ae == 
Ax = —X,P, 
ee =f 
Op 
= —X, (I— A] (A,A7)-'A,) 
Oo 
Cr — 


We may illustrate again with primal linear program (6.22). Taking x‘? = 
(2,1,5) and w = 120 yields 


v = (A7AT) | A, = (133, —68}) 
Cm — pe 
and 
r = e—wA = ( 512, 1365, 225 ) 


This r > 0, so it provides a valid bound. We may conclude that the present x has 
objective value at most x - r© = 353} above optimal. 


6.5 Dual and Primal-Dual Extensions 


Primal-Dual Interior Point Barrier Search 


The most effective commercial interior point algorithms for linear programming 
adopt a more direct approach than either of the primal methods presented so far. 
They employ primal-dual search, which purposefully changes both primal and dual 
solutions at the same time. 

Primal—dual methods depend on optimality conditions much like those dis- 
cussed for the two primal methods. This time, however, they apply to the nonlinear 
barrier model 


min ¢-x— >" In) 
j 


st. Ax=b 


Paralleling the primal and dual relationships above for the linear objective case, 


sonditions now show that interior feasible solutions x and (v, r) relate through 
2 final equation which measures the gap between their respective solution values. 


Primal-Dual Move Directions 


Primal—dual search procedures maintain a primal solution x® > 0 and a dual solution 
yr) thathasr® > 0, Each move steps in a Newton-like direction Ax) Ay“), 


Ar) constructed to improve satisfaction of conditions : 


interior feasible : 


® respectively, 


2ok 
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Notice that these move direction are of the same scaled projection format as 
earlier ones | 6.23) and As befits the primal—dual character of the search. 
however, matrix A is sometimes scaled by primal solution X, and sometimes by dual 


R,‘. Also, the vector being projected has components (1; — ers relating to the 


j 
last condition of instead of the primal objective function ec. 

We may illustrate computations for our LP of (6.22) and (6.23) from 
interior feasible solutions x” = (2,1,5), v” = (4,-1), andr™ = (2,2,5) with 
barrier multiplier w7 = 24. There 


1 
200 7 0 0 
y=) 0 110 | Rea) os 
and 
eOe@) 
My XN NESSIE) 20 
: =| 24-1-2 |=] 22 
ips xO 24—5-5 —1 
Then 
jay — x4 1? i 
Av® < —(AX;R;1AT) AR," as ( 4 
—102 
25 
peg — x 7? 
12 
1055 
Ar® <— -Ayv®A=] 212 
a 
25) 
py — 2 0? -£ 
Ax® <— R;! ~XAr® | = é 
[7 — Xn rg =u 


Primal-Dual Algorithms 


Algorithm 6C summarizes these observations in a primal—dual barrier search algo- 
rithm for linear programming. There are many variations, depending on the ster 
sizes A and barrier multipliers j, that are generated at Step 3. Experience has shows 
that the best implementations can challenge the simplex algorithm on a variety o 
LP models and significantly outperform it on many larger instances. 


so. 


Step 2: Move Direction. Construct the next move direction to improve 


satisfaction of optimality conditions | 6. 


-1 
Ave) o — (AX,R, 1A) AR?! 


Ar) <— -AvDA 
OO 
Met xy ry 
Ax) p= R;! 


() .O 
Mt —Xn' ln 


where X, and R, are diagonal matrix forms of x and r, respectively. 
Step 3: Step Size and Barrier Multiplier. Choose a compatible step size 


Step 0: Initialization. Choose any starting primal feasible interior point 
solution, x > 0, and dual feasilble (vr) with r® > 0. Also select a 
realtively large initial barrier multiplier zo, and set solution index t <— 0. 

Step 1: Optimality. If primal-to-dual solution gap ex — bv is sufficiently 
small, stop. Current point x is either optimal in the given LP, or very nearly 


My — X17; 
QO 
Mt —Xn ln 
—X,Ar? 


Aezi > O and next barrier multiplier ji. < pr. 


Step 4: Advance. Compute a new solution 


xe) en x ae Arg Axtd 
yet) e vy + het Avt) 
rtd — yO alls Aig Avery 


EXERCISES 


®1 Consider the linear program 
max 2w,+3w2 
s.t. 4w; + 3w2 < 12 


W2 <2 


W1,W2 = 0 


[6] Solve the problem graphically. 

>) [§] Determine the direction Aw of most rapid 
improvement in the objective function at any 
solution w. 


Then advance t < t+ 1, and return to Step 1. 


(c) 
(d) 
() 


(f) 
(g) 


Exercises 293 


AD 


Explain why the direction of part (b) is feasible 
at any interior point solution to the model. 

[6] Show that w® = (1,1) is an interior point 
solution. 

[6] Determine the maximum step Amax from the 
point w that preserves feasibility in the direc- 
tion of part (b). 

[8] Plot the move of part (e) and the resulting 
new point w” in the graph of part (a). 

Explain why it is easier to find a good move di- 
rection at w® than at w™. 
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6-2 Do Exercise 6-1 for the LP 


min 9w;+1w2 

st. 3w,+2w2 > 6 
2w, +3w2 > 6 
W1,W2 = 0 


and point w = (2, 3). 

6-3 The following plot shows several feasible points 
inalinear program and contours of its objective func- 
tion. 


Determine whether each of the following sequences 
of solutions could have been one followed by an in- 
terior point algorithm applied to the corresponding 
standard-form LP. 


(a) [6] P1,P5,P6 
(b) P3,P5,P6 

(c) [Q] P3,P8,P6 
(d) P3,P7,P6 

(ec) &] P3,P4,P5,P6 
(f) P2,P5,P6 


6-4 Determine whether each of the following is an 
interior point solution to the standard-form LP con- 
straints 


4x, +1x3 =13 
5x1 + 5x =15 
X1,X2,%3 > 0 

(a) x= (,0,1) 

(b) dnqpot (2195) 

(c) Bix] i279) 

(d) x = (5, 1,1) 
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(ce) Ax=@,2,) 

Gx), OF3513) 

6-5 Write all conditions that a feasible direction Aw 
must satisfy at any interior point solution to each 
of the following standard-form systems of LP con- 
straints. 


(a) BI) 2w1 +32 — 3w3 = 5 
4w, —1w2.+1w3=3 
W1, W2, W3 > 0 
(b) Tw, +1w2 -1w3 = 8 
2w, — 5w2 + 3w3 = —7 


W1,W2,W3 > 0 


6-6 Table 6.4 shows several constraint matrices A 
(or A,) of standard-form LPs and the correspond- 
ing projection matrices P (or P,). Use these results 
to compute the feasible direction for the specifed 
equality constraints that is nearest to the given di- 
rection d, and verify that the result satisfies feasible 
direction conditions at any interior point solution. 


(a) OO) x 4+2m+%3=4 andd=(,—6,3) 


—2x, +x. =-1 
(b) 3x, -x. +4x3=4 andd=(1,-1,7) 
5 — 2x3 —all 


6-7 Consider the standard-form LP 
i 14z, + 3z2 + 5z3 
st. 2z;-z3=1 
w+ =1 
Z1, Z2, 273 = 0 


min 


(a) [Q] Determine the direction of most rapid ob- 
jective function improvement at any solution z. 

(b) [§] Compute the projection matrix P for the 
main equality constraints. 

(c) [&] Apply your P to project the direction of (a). 
(d) [Q] Verify that the result of part (c) is improving 
and feasible at any interior point solution. 

(e) Describe the sense in which the direction of part 
(c) is good for improving search at any interior 

point solution. 


6-8 Do Exercise 6-7 for 


max 47, —72+7z3 
s.t. yt+73=4 
222 = 12 


Z1, 22,273 = 0 


TABLE 6.4 Projection Matrices 


AorA, PorP, 
eerie © 2a 053333: - (013333) ) —0:3333 
0 1 S11 6—03833 0.3333 0.3333 
—0.3333 0.3333 0.3333 
ie alOMe Ss 0.2540 0.1016 -0.4232 
peer) 0 “Os1016 0.0406 —-0.1693 
—0.4232 -0.1693 0.7054 
1 Pa tal: 0.0333 0.0667 —0.1667 
= 1 0 0.0667 0.1333 -0.3333 
0.1667 —0.3333 0.8333 
Sel 4 0.0816 0.2449 —-0.1224 
0 1 -2  -0.2449 0.7347 ~—-0.3673 
-0.1224 0.3673 ~—-0,1837 
12 3 4 0.0819 -0.0755  -0.1132 
OPS Bs =2 >= 010755" © 10.3019815 04528 
-0.1132 0.4528 0.6793 
4 -3 2 0.0533 -0.0710 -0.2130 
0 3 -1 -0.0710 0.0947 0.2840 
-0.2130 0.2840 0.8521 
= 2 0 0.6667 0.3333 0.3333 
allies ail 0.3333 0.1667 0.1667 
0.3333 0.1667 0.1667 


®&9 An interior point search using scaling has 
reached current solution x” = (2, 5, 1, 9). Compute 
ve affine scaled y that would correspond to each of 
ve following x’s. 
a) &) (1.1.1.1) 
>) (4,2, 3,5) 
&] (3,5, 1, 6) 
a) (2,5,1,9) 


10 Do Exercise 6-9 taking the listed vectors as 
<aled y’s and computing the corresponding x. 
11 Consider the standard-form LP 

min 2x, +3x2 + 5x3 

st. 2x; + 5x2 + 3x3 = 12 


X1,X2,x3 > 0 


*h current interior point solution x® = (2, 1,1). 


| Sketch the feasible space in a diagram like 
Figure 6.4(a) and identify both the current so- 
lution and an optimal extreme point. 

| Sketch the corresponding affine-scaled fea- 
sible space showing scaled equivalents of all 
points in part (a). 

| Derive the associated affine-scaled standard 


form| 6,10]. 
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6-12 Do Exercise 6-11 using the LP 


max 6x; + 1x2 + 2x3 


Sih: Xy+Xo+ 5x3 = 18 


X1,X2,x%3 > 0 


and current solution « = (1, 7, 2). 
6-13 Return to the LP of Exercise 6-11. 


(a) [| Compute in both x and y space the next 
move direction that would be pursued by affine 
scaling Algorithm 6A. 

(b) | Verify that the Ax of part (a) is both improy- 
ing and feasible. 

(c) | Compute the step size A that Algorithm 6A 
would apply to your move directions in part (a). 

(d) [| Plot the move of parts (a) and (c) in both 
original x-space and scaled y-space. 


6-14 Do Exercise 6-13 on the LP of Exercise 6-12. 
6-15 Consider the standard-form LP 


min 10x, + 1x2 
St. xX, —%)+2x3 =3 
X2 — X3 = 2 


X1,%2,x%3 = 0 


(a) [] Show that x = (4,3, 1) is an appropriate 
point to start affine scaling Algorithm 6A. 

(b) [] Derive the associated scaled standard form 
corresponding to solution x. 

(c) &] Compute the move direction Ax that would 
be pursued from x by Algorithm 6A (refer to 
Table 6.4 for projection matrices). 

(d) [] Show that your direction is improving and 
feasible at x. 

(ec) [| Compute the step size 4 that Algorithm 6A 
would apply to your direction, and determine 
the new point x" that results. 


6-16 Do Exercise 6-15 using the LP 


max 6x; + 8x2 + 10x3 
s.t. 9x, — 2x2 + 4x3 16 
2x2 = 2x3 =-1 
X1,%X2,%3 > 0 
and initial solution x = (4, 5,1). 
6-17 Suppose that affine scaling Algorithm 6A has 
reached current solutionx") = (3, 1, 9). Determine 
whether each of the following next move directions 
would cause the algorithm to stop and why. If not, 
compute x”), 
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(a) | Ax= ©, 2,2) 

()  Ax=(0,4,-9) 
(c) BlAx= 6. 6,0) 
@  Ax=(,-2,0) 


6-18 Consider the standard-form LP 
max 13w, —2w2.+w3 
s.t. 3w, + 6w2 + 4w3 = 12 
W1, W2,W3 = 0 


(a) [§] Sketch the feasible space in a plot like Fig- 
ure 6.6, identify an optimal extreme point, and 
show that w! = (1.4,0.7,0.9) and w®? = 
(0.01, 0.01, 2.9775). 

(b) [§] Form the corresponding log barrier problem 
with multiplier > 0. 

(c) &] Using ~ = 10, evaluate the original and 
log barrier objective functions at w” and w®. 
Then comment on the effect of barricr terms at 
points far from the boundary versus ones near 
the boundary. 

(d) & &] Use the class optimization software to 
solve log barrier form (b) with multipliers w = 
100, 10, and 1. 

(e) How does the trajectory of optimal solutions to 
part (b) evolve as w > 0? 


6-19 Do Exercise 6-18 for the LP 
min 2w;+5w2—-wW3 
st. wy + 6w2 + 2w3 = 18 
W1,W2,W3 = 0 


and points w = (6, 1, 3), w” = (0.01, 0.01, 8.965). 
6-20 Determine whether each of the following 
could be the first four barrier multiplier values 


used in the outer loop of a barrier search Algo- 
rithm 6B. 


(a) Bl 100,80,64,51.2 
(b) 100,200,400,800 
(c) BI 100,500,1,90 
(d) 600,300,150,75 


6-21 Assume that barrer Algorithm 6B has com- 
puted an appropriate move direction Ax for its 
standard-form LP and a maximum feasible step size 
Amax- For each of the original objective functions be- 
low, determine which of the following curves best de- 
picts how the corresponding barrier objective func- 
tion will vary with A € [0, Amax]. 


barrier objective 
value after step A 


barrier objective 
value after step 4 


barrier objective 
value after step 


barrier objective 
value after step 0 


eS er" 
| 
uy max 
(I) 
I 
| 
I 
I 
| 
I 
SEAS, 
r max 
(11) 
| 
| 
| 
| 
| 
| 
| 
nr 
uy max 
(III) 
| 
| 
| 
| 
| 
| 
| ae 
r max 
(IV) 


(a) [] max 34x, — 19x. — 23x3 + 4x4 
(b) min 444x, + 15x. + 1x3 + 98x. 


6-22 Consider the LP 


min 


S.t. 


4x1 — xX) + 2x3 
4x, — 3x. + 2x3 = 13 
3x2 = X3 all 


X1,X2,%3 = 0 


a) 
b) 


c) 


[9] Show that x = (3, 1,2) is an appropriate 
point to start barrier Algorithm 6B. 

IQ] Form the corresponding log barrier problem 
with multiplier 4 = 10. 

[S] Compute the move direction Ax that could 
be pursued from x by barrier Algorithm 6B. 
(See Table 6.4 for the projection matrix re- 
quired.) 

[6] Verify that your direction of part (c) is im- 
proving and feasible in the barrier model of part 
(b) at x, 

| Determine the maximum step Amax from x 
that preserves feasibility in your direction of 
part (c) and the A to be employed. 

[S] Will the barrier objective function first in- 
crease, then decrease, or first decrease, then in- 
crease, as the step A applied to your direction of 
part (c) grows from 0 to Amax? Explain. 

The next time that multiplier w is changed, 
should it increase or decrease? Explain. 


23 Do Exercise 6-22 for the LP 


max —x, + 3x. + 8x3 
s.t. x1 +2x. +43 =14 
—2x,+%x2 =3 
X1,%2,x%3 > 0 
mx”) — (1,5, 3). 
SUGGESTED READING 


= azaraa, Mokhtar S., John J. Jarvis, and Hanif Sher- 


1990), Linear Programming and Network Flows, 
>y, New York. 
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6-24 Return to the LP and x of Exercise 6-15. 


(a) 
(b) 


(c) 
(d) 


(e) 


[6] Show that x = (4, 3, 1) is an appropriate 
point to start barrier Algorithm 6B. 

Ke} Compute the move direction Ax that would 
be pursued from x® by Algorithm 6B with 
uw = 10 (refer to Table 6.4 for projection ma- 
trices). 

[0] Show that your direction is improving and 
feasible at x, 

2] Compute the maximum step size Amax that 
can be applied to your direction of part (b) at 
x without losing feasibility. 

[}] Compute the step size 4 that Algorithm 6B 
would apply to your direction, and determine 
the new point x that results. 


6-25 Do Exercise 6-24 for the LP and x of Exercise 
6-16. 


Fang, Shu-Cherng, and Sarat Puthenpura (1993), 
Linear Optimization and Extensions Theory and Al- 
gorithms, Prentice Hall, Upper Saddle River, N.J. 


( d A P T E 


Duality and Sensitivity 
in Linear Programming 


With the simplex and interior point methods of Chapters 5 and 6 we can compute 
mathematically optimal solutions to linear programming models. By comparison 
to nonlinear, discrete, and other more difficult optimization forms, this is no small 
achievement. Still, it rarely fulfills all an analyst’s needs. 

Mathematical optima do not suffice because the constant parameters from 
which they are derived—such items as costs, profits, yields, supplies, and demands— 
are almost never known with certainty at the time the model is solved. Often, they 
are not even within a factor of 10. 

How much can we trust mathematically optimal answers to very imperfectly 
parameterized models? Maybe a cost or demand controls everything. Maybe it 
could change dramatically with absolutely no impact. 

These are exactly the sorts of questions that sensitivity analysis tries to ad- 
dress. In Section 1.3 we explained that constants of OR models are really input 
parameters—values we agree to take as fixed at the system boundary in order to 
produce a tractable model. Optimal solutions simply provide a best choice of deci- 
sion variables for one fixing of the inputs. Sensitivity analysis then tries to complete 
the picture by studying how results would vary with changes in parameter values. 

Since linear programs have proved the most tractable of mathematical pro- 
zrams to numerical search, it should not surprise us that they also admit powerful 
sensitivity analysis. In this chapter we will see how post-optimality analyses of LP 
models can exploit the by-products of our search for an optimum to illuminate how 
the solution might change with variations in input variables. Remarkably in fact, we 
will see that there is an entirely separate dual linear program, defined on the same 
input constants as the main primal model, that has optimal solutions replete with 
sensitivity insights. 


7. GENERIC ACTIVITIES VERSUS RESOURCES PERSPECTIVE 


We have encountered linear programming models drawn from a wide range of ap- 
plication settings. How can we speak about the sensitivity of their optimal solutions 
to changes in input parameters when they model such different things? 
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We need a generic perspective—a standard intuition about the variables, con- 
straints, and objective functions of LP models. Then, although the details will cer- 
tainly change with the model, we can still speak in broad, common terms about the 
meaning of sensitivity results. 


Objective Functions as Costs and Benefits 


The easiest part of optimization models to interpret generically is their objective 
functions. Although the exact meaning varies greatly, virtually every model we will 
encounter can be thought about in terms of costs and benefits. 


ab. i oe Optimiza 


Choosing a Direction for Inequality Constraints 


Now consider inequality constraints. What does a > inequality sign typically signify 
Is a < sign any different? 

Every beginning algebra student knows that there is no mathematical distine- 
tion. For example, the gasoline demand constraint of the Two Crude refining mode! 
(2.3) can be written equivalently as either 


0.3x; + 0.4x2 > 2.0 or — 0.3x; — 0.4%. < —2.0 


Still, the first certainly depicts more clearly the idea that output must meet or exceed 
demand. 

Most constraints have a “natural” format of this sort. No absolute principle 
tells us which direction is most intuitive, but a rule of thumb covers the foregoing 
and most other examples: 


Inequalities as Resource Supplies and Demands 


At the level of intuitive understanding, we see that < inequalities do differ from 
> inequalities. To assign a generic meaning, let us review some of the many < 
inequalities that we have encountered: 


e x; <9 restricts the supply of Saudi petroleum in the Two Crude model (2.6) in Sectios 
Dale 


e 4x, + 2x2 < 4800 in model (5.1) (Section 5.1) limits the wood supply in the Top Brass 
Trophy example. 


e@ 0.25 (Z1/4,4,8 + 21/4,4,c + 21/4,8,c) + 0.40(21/2,4,8 + 21/2,4,c + 21/2,8,c) < 4500 enforces the 
supply limit on pressing capacity in the CFPL model of Table 4.5 in Section 4.3. 


e yu+ yi < 20 fixes the availability of overtime in the ONB example of Table 4.7 
Section 4.4. 


Common elements are a resource and a limited supply of that resource. 
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The corresponding interpretation of > inequalities is as output demands. 


e 0.4x, +0.2x2 > 1.5 in model (2.6) (Section 2.1) requires jet fuel output to meet demand 
in the Two Crude refining example. 

e 0.120x; + 0.011x. + 1.0x5 > 10 set the minimum acceptable level of chromium in the 
blend of the Swedish Steel model (4.4) (Section 4.2). 

© y1/8,A,.8 + Y1/3,B,B + Y1/8,C,B — 21/2,4,B — 21/2,A.C — 21/2,8,c = 0 demands production of 
z-inch veneer to exceed consumption in the CFPL example of Table 4.5 (Section 4.3). 

© Lyi + 1x13 +.0.8z29 — wa, > 8 enforces the demand that all checks be processed by 22:00 
(10 P.M.) in the ONB model of Table 4.7 (Section 4.4). 


As with < inequalities, the common element is a commodity or resource, but the 
direction is different. 


Equality Constraints as Both Supplies and Demands 


Equality constraints are hybrids. Consider, for example, the Swedish Steel model 
+.4) constraint: 


Xy + X2+%34+X%4+X5 + X56 + x7 = 1000 


=nforcing the requirement that ingredients in the melting furnace charge should have 
2 total weight of exactly 1000 kilograms. This (or any other) equality can just as well 
de expressed as two opposed inequalities: 


Xy +X. +%34+X4+X5 +x6 + x7 < 1000 
and 
xX, +X. +%3 4X4 +xX5 +x +x7 > 1000 
Thinking of equalities in this way makes it easy to see the hybrid. 


7.5_| Optimization model equality constraints usually can be interpreted as im- 
posing both a supply restriction and a demand requirement on some commodity _ 


or resource. 


The effect is a mixture of both limits on supply of a resource and demand for its 
utput. 


\ariable-Type Constraints 
-ariable-type limitations such as nonnegativities are constraints in a linear program. 
‘= can even stretch principle to think of nonnegativities as demands for some 
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“positiveness” resource. Still, it usually makes more sense to keep variable-type 
constraints separate. 


ign restriction constraints are usually best in- 
ble ype rather than ey or demand limits 


on resources. — 


Variables as Activities 


Turning now to the decision variables in an optimization model, let us again recall 
some examples: 


e x; in the Two Crude model (2.6) (Section 2.1) chooses the amount of Saudi crude to be 
refined. 

© Xpm in the TP model (4.5) (Section 4.3) decides the amount of product p produced at 
mill m. 

e z;, in the ONB model of Table 4.7 (Section 4.4) sets the number of part-time employees 
starting at hour h. 

e x; in the Swedish Steel model (4.4) (Section 4.2) establishes the number of kilograms of 
ingredient j used in the blend. 


The common element shared by these and other examples is a sense of activity. 


LHS Coefficients as Activity Inputs and Outputs 


Summarizing, our generic linear program chooses activity levels of appropriate sige 
to minimize cost or maximize benefit, subject to < supply limits on input resources 
> demand requirements for output resources, and = constraints doing both. 
follows immediately how we should think about the objective function and constrain® : 
coefficients on decision variables: 


- Nonzero objective function and constraint coefficients on LP decision 
variables display the impacts per unit of the variable’s activity on resources or 
commodities associated with the objective and constants 


Sometimes those impacts become clearer in block diagrams such as those o 
Figure 7.1. Each variable or activity forms a block. Inputs and outputs to the bloc! 
indicate how a unit of the activity affects the constraints and objective. For example 
each unit (thousand barrels) of Saudi petroleum refined in the Two Crude exampl< 
(2.6) inputs 1 unit (thousand barrels) of Saudi availability and 20 units (thousand 
dollars) cost to output 0.3 unit (thousand barrels) of gasoline, 0.4 unit (thousané 
barrels) of jet fuel, and 0.2 unit (thousand barrels) of lubricants. 

The second diagram in Figure 7.1 comes from the CFPL model of Table 45 
Peeling “good”-quality logs from supplier 1 as i-inch green veneer consumes los 
availability and cost to obtain specified amounts of different veneer grades. 


1000 barrels 
Saudi availability 
$20,000 


cost 


1 log of supplier 1 


good" availability 


$340 Canadian 
cost 


1 imprinting machine 


11 A.M.-12 noon 


| imprinting machine 
i2P.M. 
| imprinting machine 


S4PM. , 


| imprinting machine 
7-3 P.M. 


590 cost 


7.1 Generic Activities versus Resources Perspective 


Two Crude Example 2.1 
thousand barrels 
of Saudi 
petroleum 


processed 


(x) 


CFPL Example 4.4 
peeling 
"good"-quality 
log from supplier 
linto 4-inch 
green veneer 


(Wo, 1,116) 


ONB Example 4.5 
full-time 


check processor 


Starting 11 A.M. 


(41) 


FIGURE 7.1 Inputs and Outputs for Activities in Various Models 


0.3 unit (thousand barrels) 


gasoline 


0.4 unit (thousand barrels) 


jet fuel 


0.2 unit (thousand barrels) 


lubricants 


400 sq ft -inch 
A green veneer 


700 sq ft -inch 


B green veneer 


900 sq ft *,-inch 


C green veneer 


1000 checks processed 


11A.M.-12 noon 


° 
° 


1000 checks processed 


1-2 P.M. 
1000 checks processed 


3-4P.M. , 

1000 checks processed 
7-8 P.M. 

5—unit of 11 A.M. shift 


overtime potential 


Finally, we have an activity from the ONB model of Table 4.7. Using a full- 
“me employee on the 11 A.M. shift in the ONB example inputs cost and equipment 
‘or each on-duty hour to output check processing through the shift and overtime 
ootential. Each input or output produces a nonzero model coefficient. 
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coefficients in the following linear program: 


max +13x, +24x. +5x3 +50x4 


s.t. + 1x) + 3x2 > 89 
—3x3 —5x4 < —60 

+10x, +6x2 +8x3  +2x4 < 608 

+ 1x5 +1%x4 < 28 


Mig. rep anen O) 


Analysis: We begin by reversing the direction of the second inequality to make its 
right-hand side nonnegative (principle Ny: 


max +13x, +24x. +5x3 +50x4 


s.t. +1x, +3x. > 89 
+3x3 +5x4 > 60 

+10x, +6x2 + 8x3 +2x4 < 608 

+ 1x + 1x4 < 28 


Xo: eka 0 
We may now interpret the model as one of deciding the optimal level of 4 
activities corresponding to the 4 decision variables (principle ). The objective 


function maximizes the benefit derived from these activities (principle |7.1|) with 
coefficients showing the benefit per unit activity (principle DE 

The 4 main constraints deal with 4 commodities or resources. The initial 
being of > form, specify a demand of 89 for the first commodity and 60 for the 
second (principle| 7.4 |). The last 2, being of < form, restrict the supply of commod:- 
ties 3 and 4. Nonnegativity constraints merely enforce the variable type (principle 
[76}). 

Coefficients on the left-hand side of main constraints show inputs and output 
per unit of each variable’s activity (principle ). Specifically, each unit of activity 
1 consumes 10 of commodity 3 to produce 1 of commodity 1; each unit of activit 
2 consumes 6 of commodity 3 and 1 of commodity 4 to produce 3 of commodity 1 
each unit of activity 3 consumes 8 of commodity 3 to produce 3 of commodity 2; ane 
each unit of activity 4 consumes 2 of commodity 3 and 1 of commodity 4 to produce 
5 of commodity 2. 


3 £4 QUALITATIVE SENSITIVITY TO CHANGES IN MODEL 
COEFFICIENTS 


Armed with a generic way of thinking about LP models, we are ready to consid 
the sensitivity of optimization model results to changes in their input parameters 
constants. Let us begin qualitatively. Much can be learned from looking just at th: 
directions of change rather than their magnitudes. 


7.2 Qualitative Sensitivity to Changes in Model Coefficients 


Relaxing versus Tightening Constraints 


Consider first relaxing versus tightening constraints. Figure 7.2 shows the idea graph- 
ically. A two-variable model in part (a) has the feasible region indicated by shading. 
Relaxing a constraint as in part (b) admits new feasible solutions. The corresponding 
optimal value must stay the same or improve. On the other hand, if the constraint 
is tightened as in part (c), fewer feasible solutions are available. The optimal value 
can only stay the same or worsen. 


7.9__| Relaxing the constraints of an optimization model either leaves the op- 
timal value unchanged or makes it better (higher for a maximize, lower for 


changed or makes it worse. _ 


Principle is the most broadly applicable of all sensitivity insights. It holds 
for any optimization model—LP or not—and any constraint type. 
Swedish Steel Example Revisited 


For a more concrete illustration, we revisit our Swedish Steel blending example of 
Section 4.2. The associated LP model is 


min 16x; + 10x2 + 8x3 + 9x4 + 48x5 + 60X65 + 53x7 (cost) 

St. xy bay tx3 4x4 4x5 +46 +247 = 1000 (weight) 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 > 6.5 (carbon) 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 < 7.5 
0.180x1 + 0.032x2 + 1.0x5 > 30.0 (nickel) 
0.180x; + 0.032x2 + 1.0x5 = BND 
0.120x; + 0.011x2 + 1.0x6 > 10.0 (chromium) 
0.120x, + 0.011x2 + 1.0x¢ = 12.0 
0.001x2 + 1.0x7 > 11.0 (molybdenum) 
0).001x2 + 1.0x7 < 13.0 
Rus TS (availability) 
x2 < 250 
X1,...,47 = 0 (nonnegativity) 


(7.1) 
Effects of Changes in Right-Hand Sides 


Figure 7.3 plots the optimal value in the Swedish Steel model (7.1) versus two of its 
right-hand-side coefficients. Specifically, part (a) follows the impact of the right-hand 
side (RHS) for the scrap 1 availability constraint 


x <75 (7.2) 


assuming that all other parameters are held constant. Similarly, part (b) tracks 
changes with the right-hand side of the minimum chromium content constraint: 


0.120x; + 0.011x2 + 1.0x5 > 10.0 (7.3) 


infeasible cases of this minimize cost problem are plotted as infinite costs. 
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(a) Original (b) Relaxed (c) Tightened 


FIGURE 7.2 Effects of Relaxing Versus Tightening Constraints 


optimal 
value 
{slope —o (infeasible) 


slope —4.98 


9700 - 
We slope —3.38 
ae current 9526.9 a e aoe 0.00 
9500 ec 
ly cal 
(a sll | 
es en eee 
Shoal i Lo 
Ss ioe) 


(a) Supply ($) constraint case (scrap 1 availability) 


optimal 
value 


current 9526.9 


slope 8.57, 


slope 0.00 


0.0 


(b) Demand (2 ) constraint case (minimum chromium content) 


FIGURE 7.3 Sensitivity of Swedish Steel Optimal Value 
to Right-Hand Sides 


In Section 7.7 we discuss how to generate such plots. For the moment, just notice 
how the trends differ. Increases in the scrap 1 right-hand side help the minimize cos* 
objective, and decreases hurt. Changes in the minimum chromium right-hand side 
produce exactly the opposite effect. 


RG oc 200 a Ot. in ew hs oe ee oe Lee 
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This apparently contradictory experience follows from one of the constraints 
being of < supply form and the other a > demand. An increase in the right-hand 
side for the two forms produces different effects on the feasible set. 


ae _Changesi in n LP model neu tandaidea coefficients affect the feasible: space 
as follows: | oe 


_ Constraint =RHS  —_RHS 
‘Type Increase Decrease 


Supply (s) Relax Tighten 


_ Demand () “Tighten Relax. 


The plot in Figure 7.3(a) declines because an increase in the right-hand side 
relaxes a < constraint. Under principle the optimal value must stay the same 


or decrease (improve). Increases in the RHS of part (b) tighten the corresponding 
> constraint. The optimal value must either stay the same or increase (worsen). 


SAMPLE EXERCISE 7.2: QUALITATIVELY ASSESSING RHS SENSITIVITY 


Determine for each of the following objective function and constraint pairs the qual- 
itative effect of increasing or decreasing the constraint right-hand side. Assume that 
the given constraint is not the only one. 


(a) max 13w, —1lw2+w3 (b) max 13w; —11w2+W3 
s.t. Ow, + wo — w3 < 50 S.C. 9w, +wW2 —w3 = 50 
(c) min 8z; —4z2. + 15z3 (d) min 8z, —4z. + 15z3 
st. 62, —3z2 < -19 st. 6z, —3z2 > -19 


Analysis: We apply principles and : 


(a) A RHS increase relaxes this < constraint, meaning that the optimal value will 
stay the same or increase (improve for a maximize objective). RHS decreases have 
the opposite effect. 


(b) A RHS increase tightens this > constraint, meaning that the optimal value will 
stay the same or decrease (worsen). 


c) A RHS increase relaxes this < constraint, meaning that the optimal value will 
stay the same or decrease (improve for a minimize objective). 


d) A RHS increase tightens this > constraint, meaning that the optimal value will 
stay the same or increase (worsen). 


Effects of Changes in LHS Constraint Coefficients 


With linear programs, principle holds just as well for coefficient changes on the 
left-hand side (LHS) of a constraint as for RHS variations such as those in Figure 


> 


7.3. Since constraint functions are just weighted sums of the variables, minimum 


308 Chapter Seven e Duality and Sensitivity in Linear Programming 


chromium content constraint (7.3) is relaxed if we increase the 0.120 yield coefficient 
on x, to, say, 0.500. It is tightened if, for instance, we reduce it to —0.400. A larger 
coefficient on the left-hand side of a > constraint makes it easier (for nonnegative 
variables) to satisfy, and a smaller coefficient makes it harder to satisfy. Other cases 
are similar. 


7. i Changes i in LP model LHS constraint coefficients on nonnegative deci- 


Coefficient Coefficient 


_ Increase Decrease 
Supply(<) Tighten «Relax 


_Demand() Relax Tighten 


SAMPLE EXERCISE 7.3: QUALITATIVELY ASSESSING LHS SENSITIVITY 


Return to the objective functions and constraints of Sample Exercise 7.3, and de- 
termine the qualitative impact of each of the following coefficient changes. Assume 
that all variables are required to be nonnegative. 


(a) Change the coefficient of w2 to 6 in Sample Exercise 7.2(a). 
(b) Change the coefficient on w; to 0 in Sample Exercise 7.2(b). 
(c) Change the coefficient on z; to 2 in Sample Exercise 7.2(c). 


(d) Change the coefficient on z3 to —1 in Sample Exercise 7.2(d). 


Analysis: We apply principles and : 


(a) The change from 1 to 6 is an increase, meaning that the < constraint is tightenee 
and the maximize optimal value will stay the same or decrease. 


(b) The change from —1 to 0 is an increase, meaning that the > constraint is relaxed 
and the maximize optimal value will stay the same or increase. 


(c) The change from 6 to 2 is a decrease, meaning that the < constraint is relaxec 
and the minimize optimal value will stay the same or decrease. 


(d) The change from 0 to —1 is a decrease, meaning that the > constraint is tightenec 
and the minimize optimal value will stay the same or increase. 


Effects of Adding or Dropping Constraints 


Principle | 7.9| can even be stretched to cases where we change the model more 


7.2 Qualitative Sensitivity to Changes in Model Coefficients 


It follows that adding constraints can only make the optimal value worse. Dropping 
constraints can only make it better. 

Figure 7.3(a) implicitly includes the dropping case. Elimination of the scrap 
1 availability constraint is the same thing as making its right-hand side very large. 
Part (a) of the figure shows how the optimal value declines, an improvement for a 
minimize cost problem. 


Consider the linear program 


max 6y; + 4y2 


st. yi tye <3) 
Y1 <2 
y2 <2 


y1,y2 20 
Assess the qualitative impact of each of the following constraint changes. 


a) Dropping the first main constraint 


5) Adding the new constraint y2 > y; 


Analysis: We apply principles and : 


a) Dropping a constraint relaxes the feasible set. Thus the maximize optimal value 
2an only stay the same or increase. 


>) Adding a new constraint tightens the feasible set. Thus the maximize optimal 
-alue can only stay the same or decrease. 


Effects of Unmodeled Constraints 


fen in OR studies the possibility of adding constraints arises in the context of 
snmodeled phenomena. For example, managers in the Swedish Steel model (7.1) 
might prefer to use all or none of the 250 kilograms of scrap 2 instead of trying to 
easure amounts in between. 
Such a requirement is discrete. (Do you see why?) Thus there are good reasons 
»r modelers to neglect it in the interest of tractability. But how would results change 
*such a difficult-to-express limit on solutions had been modeled? 


7.13 Explicitly including previous! 
model must leave the optimal ve 


Changing Rates of Constraint Coefficient Impact 


_ook again at Figure 7.3. Besides the fact that tightening constraints leaves the op- 
“mal value the same or worse, there is another pattern. The more we tighten, the 
seater the rate of damage. In part (a), for instance, tightening means reducing the 
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RHS coefficient. It has no impact at all for right-hand sides above 83.3. As we con- 
tinue to squeeze the feasible set, however, the rates of change increase. Ultimately. 
the model becomes infeasible. 

Relaxing constraints produces an opposite effect. For example, in Figure 7.3(b 
relaxing means reducing the RHS. The figure shows a steep rate of decline at higher 
values of the right-hand side that dimishes as we make the coefficient smaller. Every 
linear program exhibits similar behavior. 


7.14 | Coefficient changes that help the optimal value in linear programs by re- 
laxing constraints ste less and less as the pies becomes large. ees that 


Model Form 
(Primal) 


Supply Demand 


optimal 
value 


Maximize objective 


optimal : optinal ees : 
value : value - . 
: i 


Minimize objective 


=-RHS 1. rs 


Note that principle is limited to linear programs. Models with nonconye 
feasible sets (see Section 3.4) may behave differently. 

To get some intuition about [7.14], think of yourself as struggling heroicalh 
to force solutions to conform to your favorite supply or demand requirement. The 
more you tighten the constraint you control, the more it becomes the driving force 
behind all decisions; its impact becomes greater and greater. On the other hand. © 
you are driven back and forced to relax your constraint, there are many others + 
take up the cause; you make less and less difference. 


—_— 


7.2 Qualitative Sensitivity to Changes in Model Coefficients Sid 


SAMPLE EXERCISE 7.5: QUALITATIVELY ASSESSING RATES OF CONSTRAINT 
y CHANGE 


Determine whether rates of optimal objective function value change would be steeper 
or less steep with the magnitude of each of the following RHS changes. 


(a) Increase the RHS of 4y; — 3y2 < 19 in a maximize linear program. 


(b) Decrease the RHS of 3y; + 50y2 > 40 in a maximize linear program. 


DS 


(c) Decrease the RHS of 14y; + 8y2 < 90 in a minimize linear program. 


(d) Increase the RHS of 3y; — 2y2 => 10 in a minimize linear program. 


Analysis: 


(a) Under principle 710, the optimal value will improve (become larger) with 


increases in this RHS because the < constraint is being relaxed. Principle 
implies that the rate of improvement should stay the same or diminish. 


b) Under principle (7.10 |, the optimal value will improve (become larger) with 


decreases in this RHS because the > constraint is being relaxed. Principle 
implies that the rate of decline should stay the same or diminish. 


c) Under principle , the optimal value will worsen (become larger) with in- 


creases in this RHS because the < constraint is being relaxed. Principle implies 
that the rate of improvement should stay the same or steepen. 


d) Under principle 7.10 |, the optimal value will worsen (become larger) with in- 


creases in this RHS because the > constraint is being tightened. Principle 
implies that the rate of decline should stay the same or steepen. 


Effects of Objective Function Coefficient Changes 


Objective function coefficients neither relax nor tighten constraints, but we may still 
oe very interested in their impact on results. Figure 7.4(a) plots the optimal value 
n the (minimize cost) Swedish Steel model (7.1) as a function of the unit cost for 
scrap 4 (coefficient of x4). Part (b) tracks the optimal value in the (maximize margin) 
CFPL model in Table 4.5 versus the selling price for sheets of $-inch AC plywood 
coefficient of Z1/2,a,c). 

When an objective function coefficient multiplies a nonnegative variable, as 
most of them do, the direction of change is easy to predict. For example, decreasing 
the unit cost of an activity certainly makes lower-cost solutions easier to find; the 
»ptimal value could only improve. 


7.15 | Changing the objective function coefficient ofa nonnegative variable in 
an optimization model affects the optimal value as follows: 


-Coelficient — Coefficient 
(Primal) _ Increase Decrease 


Maximize objective Same or better Same or worse _ 
Minimize objective Same or wor: eorbetter 
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(b) Max case (CFPL selling price for 41-inch AC plywood) 


FIGURE 7.4 Sensitivity of Optimal Values to Changes in Objective Function Coefficients 


For each of the “olla Opiccine functions, determine the Anes impaaay of 


the indicated change in objective function coefficient. Assume that all variables ar 
nonnegative. 


7.2 Qualitative Sensitivity to Changes in Model Coefficients 


(a) Change the coefficient of w; to 7 in max 12w; — w2. 
(b) Change the coefficient of w2 to 9 in max 12w; — 

(c) Change the coefficient of w; to 60 in min 44w; + 3w. 
(d) Change the coefficient of w2 to —9 in min 44w, + 3w. 


Analysis: We apply principle : 


(a) A change from 12 to 7 is a decrease, implying that the maximize optimal value 
will stay the same or worsen (decrease). 


(b) A change from —1 to 9 is an increase, implying that the maximize optimal value 
will stay the same or improve (increase). 


(c) A change from 44 to 60 is an increase, implying that the minimize optimal value 
will stay the same or worsen (increase). 


(d) A change from 3 to —9 is a decrease, implying that the minimize optimal value 
will stay the same or improve (decrease). 


Changing Rates of Objective Function Coefficient Impact 


Readers who absorbed the discussion above about rates of optimal value change 
caused by variations in constraint coefficients should be surprised by the shapes of 
curves in Figure 7.4. Consider, for instance, the maximize case of part (b). A move to 
the right, which increases the coefficient of Z1/2,a,c, helps the objective function—just 


as we concluded in 

What is different is that the more we increase the coefficient, the more rapidly 
the optimal value improves. In contrast to the decreasing returns we saw with con- 
straint changes that helped the optimal value, favorable objective function changes 
produce increasing rates of return. 


7.16 . Objective function coefficient changes that help the optimal value in lin- _ 
ear programs help more and more as the change becomes large. Changes that _ 
hurt the optimal value hurt less and less. The result is is Sensitivity es shaped 
like the following: oe 


Maximize Objective = Minimize Objective 
(Primal) = ao 


optimal 


optimal 2 
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As with , principle is limited to linear programs with their convex feasible 
sets. Nonconvex optimization models may perform differently. 


A full explanation of the incongruity between rates of change in constraint and 
objective function coefficients must await the duality development of the next few 
sections. For now, return to your struggling hero role, this time by manipulating the 
unit cost or benefit of some activity. Changing your objective function coefficient in 
the direction that helps the optimal value draws more and more action to your activity. 
The more it dominates the solution, the more its objective function coefficient affects 
the optimal value. On the other hand, changes that hurt the optimal value are 
cushioned by transfer of responsibility to other activities. They have less and less 
impact as the change becomes large. 


_ SAMPLE EXERCISE 7.7: QUALITATIVELY ASSESSING RATES OF OBJECTIVE _ 


Determine whether the objective function coefficient change indicated would in- 
crease or decrease the optimal objective function value in each of the following. 
and indicate whether the rate of change will steepen or become less steep with the 
magnitude of the change. Assume that all variables are nonnegative. 


(a) Increase 42 on x; in max 42x, + 13x. — 9x3. 
(b) Decrease —9 on x3 in max 42x, + 13x. — 9x3. 
(c) Increase —2 on x2 inmin 12x; — 2x2 + x4. 

(d) Decrease 0 on x3 in min 12x, — 2x7 + x4. 
Analysis: 


(a) The optimal value improves (increases) with any objective function coefficient 


increase in a maximize model (principle ). Thus, under principle the rate 


of increase should stay the same or steepen. 


(b) A decrease would be to c; < — 9. The optimal value worsens (decreases) with 
any objective function coefficient decrease in a maximize model (principle ). 
Thus, under principle | 7,16 | the rate of increase should stay the same or lessen. 


(c) The optimal value worsens (increases) with any objective function coefficient 
increase in a minimize model (principle ). Thus, under principle the rate 
of increase should stay the same or lessen. 

(d) The optimal value improves (decreases) with any objective function coefficient 


decrease in a minimize model (principle ). Thus, under principle 
rate of increase should stay the same or steepen. 


Effects of Adding or Dropping Variables 


The final sort of sensitivity analysis we need to consider is the adding or dropping of 
activities (which is implemented mathematically as adding or dropping of decision 
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variables). For example, what happens if we decide to consider a new scrap source 
in the Swedish Steel model (7.1)? What happens if we drop scrap 4? 

Adding activities offers new choices; dropping them reduces possibilities. The 
direction of optimal value change should then be apparent. 


7, = ‘Adding op imization model activities (variables) must leave the optimal 


changed or degraded. 


SAMPLE EXERCISE 7.8: QUALITATIVELY SSESSING ‘ DDED AND DROPPED - 


: VARIABLES > 


Determine the qualitative impact of adding or dropping a variable in linear programs 
with each of the following objective functions. 


(a) max 27y; — y2 + 4y3 
(b) min 33y, + Lly2 + 39y3 
Analysis: We apply principle : 


(a) Adding a variable in this maximize LP can only help, leaving the optimal value 
the same or higher. Dropping a variable would make it the same or lower. 


(b) Adding a variable in the minimize LP can only help, leaving the optimal value 
the same or lower. Dropping a variable would make it the same or higher. 


7.3 QUANTIFYING SENSITIVITY TO CHANGES IN LP MODEL 
COEFFICIENTS: A DUAL MODEL 


All the LP sensitivities of Section 7.2 related to directions of changes rather than 
magnitudes. It is certainly helpful to know whether, say, a right-hand-side modifica- 
tion will help or hurt the optimal value. Still, we would gain much more insight if we 
could quantify the rates of change. 

The OR approach to computing quantities we would like to knowis to represent 
them as variables, formulate a model of how they interrelate, and solve the model to 
obtain their values. In this and Section 7.4 we pursue that approach. More specifically, 
we define new sensitivity variables and formulate the conditions that a quantitative 
sensitivity analysis should fulfill. 


Primals and Duals Defined 


With one model being used to analyze sensitivity of another, it is important to dis- 
tinguish between primal and dual. 


All the linear programs formulated in earlier chapters were primals. 


oD. 
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Dual Variables 


To begin deriving the dual, take another look at the Figure 7.3 plots of the (primal 
Swedish Steel model’s optimal value as a function of right-hand side. Dual variables 
quantify slopes of such curves. 


pia: 


- There is one dual variable for each main primal constraint. Each reflects 
i i € per unit increase from the given 


right-hand-side value of the corresponding constraint. 


We often denote the dual variable on primal main constraint i by vj. 

The right-hand side of primal constraint (7.2) is 75. Figure 7.3(a) shows the 
rate of change in the primal optimal value, and thus the value of the dual variable 
corresponding to this availability constraint is —3.38 kroner/kilogram. 

Of course, we would prefer to know such slopes for all possible values of right- 
hand sides. For instance, in Figure 7.3(a) we have 


+oo between —oo and 0 
—4,98 between 0 and 60.42 
—3.38 between 60.42 and 83.33 

0 between 83.33 and +c 


However, several LPs have to be solved to derive such plots. 
Since the dual only provides subsidiary analysis of the primal, which is the mode 


of real interest, definition adopts a more manageable standard. Optimal dua! 
variable values for any single LP yield rates of change only at the original RHS value 
For instance, only the —3.38 rate at RHS = 75 would be obtained for the constraint 
of Figure 7.3(a). 

This definition is somewhat unclear if the RHS value happens to fall at exactly 
one of the points where slopes change. For example, if the given RHS in Figure 
7.3(a) had been 83.33, the corresponding dual variable value could come out eithes 
—3.38 or 0.00 (or anything in between). The first applies to the left of 83.33 and the 
second to the right. At exactly 83.33 the slope is ambiguous. 


SAMPLE EXERCISE 7, 9: “UNDERSTA ING DUAL VARIABLES _ 


Refer to Figure 7.3(b)’s plot of sensitivity to minimum chromium constraint (7.3). 
(a) What is the corresponding optimal dual variable value? 
(b) What would it be if the constraint RHS had been 7.0? 


(c) What would it be if the constraint RHS had been 9.0? 
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Analysis: 


a) By definition the dual variable corresponding to this minimum chromium 
constraints should equal the slope of the sensitivity curve at RHS = 10.0. That is, the 
dual variable has value 36.73 kroner/kilogram. 


b) For RHS = 7.0 the corresponding dual variable value or rate of change is 8.57 
kroner/kilogram. 


(c) At RHS = 9.0 the dual variable value is ambiguous. Computation might produce 
either the 8.57 to the left or the 36.73 to the right. 


Dual Variable Types 

We want to develop constraints and conditions that dual variables must satisfy to 
provide meaningful sensitivity information. Qualitative principles and 
already yield sign restrictions on rates of change in inequality right-hand sides. For 
example, an increase in the RHS of a < constraint in a minimize model relaxes the 
constraint, which leaves the optimal value unchanged or reduced. The corresponding 
vy <0. 


Primal fiss iis> iis= 


Minimize objective v,<0Q v,20 Unrestricted 
Maximize objective v,20 v,<0 Unrestricted 
ee 


Notice that principle specifies dual variables for = constraints to be un- 
restricted in sign (URS). Since an equality can act as either a < or a > (principle 
7.5|), the corresponding dual rate of change can be either positive or negative. 


Two Crude Example Again 


To have an easy example to follow, let us return to the familiar Two Crude refining 
model of Section 2.1. That primal linear program was 


min 20x, +4+15x2 


st. 0.3x, +0.4x2 > 2 Vy (gasoline demand) 
0.4x, +0.2x. > 1.5 SV (jet fuel demand) 
0.2x; +0.3x2. > 0.5 V3 (lubricant demand) (7.4) 
Lx, <9 :V4 (Saudi availability) 
+1x. <6 Ds (Venezuelan availability) 
M4, 42 = 0 


The unique primal optimal solution is xf = 2, x} = 3.5, with optimal value 92.5. 
Notice that we have assigned dual variables to each main constraint. Applying 


orinciple 
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become 
¥y 20) Sg AO v3 = ON ve = 0) is =O (7.5) 


Increasing > right-hand sides in a minimize problem can only raise the optimal cost 
or leave it unchanged. Increasing those of <’s can only lower it or leave it unchanged. 


ables indicated should be nonnegative, nonpositive, or of URS type. 


(a) min +5x, + Ix. +43 +5x4 


st. +1x, +4x. +2x3 —236 Vy 
+3x, +2x. +8x3 +2x4 < 250 a) 

—5x, —2x. +1x3 4+1x, <7 V3 

+1x3 +1x, > 60 D4 


X41,X2,%3,%4 = 0 


(b) max +13x, +24x. +5x3 +50x4 


s.t. + 1x + 3x2 > 89 DVy 
+1x3 +5x, > 60 Ivo 

+10x, +6x. +4+8x3 +2x, <608 V3 

+1x> +1x, =28 WA 


X1, X2, X3,X%4 > 0 


Analysis: We apply rule , 
(a) Vj URS, v2 <0, v3 < 0, V4 => 0 
(b) v1 <0, V2 0, v3 > 0, v4 URS 


Dual Variables as Implicit Marginal Resource Prices 


Because the dual variables tell us the rate of change in the optimal value for another 
unit of each RHS, they provide a sort of implicit price on the resource of each con- 
straint model. To be more precise, they yield what economists call marginal prices. 


In Two Crude model (7.4), for instance, the first constraint models gasoline 
demand. Variable v1, which is in units of thousands of dollars per thousand barrels. 
will tell us the implicit price of gasoline at the margin when RHS = 2000 barrels 
are demanded, that is, how much the last 1000 barrels cost to produce in the opti- 
mal operating plan. Similarly, variable v4 reflects the marginal impact of the Saudi 
availability constraint at its current level of 9000 barrels, which is the implicit value 
of another 1000 barrels of Saudi crude. 
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] 11: INTERPRETING D 


The Top Bass Trophy example model (5. 1) of Section 5.1 is 


max 12x; + 9x2 [profit (dollars) 
st. x4 ; < 1000 :Vy (brass footballs) 
x2 < 1500 V2 (brass soccer balls) 


Xi + x2 < 1750 V3 (brass plaques) 
4x, +2x,  < 4800 V4 [wood (board feet) | 
x1, X%2 > 0 


Interpret each of the 4 corresponding dual variables and determine their signs and 
units. 


Analysis: Applying interpretation , vy is the marginal value or contribution to 
profit of brass footballs at the current 1000 availability level, that is, how much we 
would pay for another one. Under principle ae, v; > 0, and its units are dollars 
per brass football. Similarly, v2 > 0 is the marginal value of brass soccer balls at 
the current 1500 availability level (in dollars per soccer ball), v3 > 0 is the marginal 
value of brass plaques at the current 1750 availability (in dollars per plaque), and 
vs = Ois the marginal value of wood at the current 4800 availability (in dollars per 
board foot). 


Implicit Activity Pricing in Terms of Resources Produced 
and Consumed 


The activity associated with any decision variable both consumes and creates con- 
straints’ resources (review Figure 7.1). We also know that the activity’s nonzero 
constraint coefficients can be interpreted as amounts consumed or created per unit 
of the activity (principle 7s ). 

By summing those coefficients times the implicit price v; of the resources in- 
volved, we can obtain an implicit marginal value (minimize problems) or price (max- 
mize problems) for the entire activity. 


Venezuelan activity 7 = 2 in the Two Crude example illustrates. Given resource 
prices v1,..., Vs, each unit of Venezuelan crude activity is implicitly worth 


5 

S> aiavi = 0.401 + 0.2v2 + 0.303 + Lvs 

i=l 
Noting sign restrictions (7.5), the first three terms measure the positive contribution 
»t fulfilling demand constraints. The last deducts (v5 < 0) the implicit worth of the 
svailability resource consumed. 
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Brass Trophy model reviewed in Sample Exercise 7.11. 


Analysis: In the Top Brass example, x; represents production of football trophies 
and x2, production of soccer trophies. Under principle | 7.23], the implicit price or 
marginal cost to produce a football trophy is the sum of coefficients on x, times the 
marginal value of the associated commodities, that is, 


4 
2S 41Vi = 1vy + 1v3 + 4v4 
i=l 
This expression simply sums the marginal cost of the 1 football, 1 plaque, and = 
board feet of wood needed to make a football trophy. Similarly, the marginal cost of 
producing a soccer trophy is 


4 
¥F Qi2Vi = 1v2 +13 + 2V4 
i=1 


Main Dual Constraints to Enforce Activity Pricing 


The notion of implicit activity worth per unit also leads to the main dua! 
constraints. If dual variables v; are really to reflect the value of constraint resources 
in the primal optimal value, the implied activity worths must be consistent with 
explicit unit costs or benefits in the primal objective function. 

To be specific, dual variables in a minimize (cost) problem would overvaluc 
an activity if they priced it above its true cost coefficient cj. An activity implicitly 
worth more than it costs at optimality should be used in greater quantity. But then we 
would be improving on an optimal solution—an impossibility. Similarly for maximize 
(benefit) problems, we would want to reject any v;’s that priced the net value of 
resources that an activity uses below its real benefit cj. Otherwise, it too would yiels 
an improvement on an already optimal solution. 


lem main dual const ints for x; > 
marginal cost of each 


Illustrating again with the Two Crude model (7.4), principle leads to ome 
constraint for primal variable x, and another for x2. . 
0.3v, + 0.4v2 + 0.2v3 + Iv4 < 20 
0.4y; + 0.2v2 + 0.3v3 + lvs < 15 


(76 


Without such limits, say if the worth evaluation on the left-hand side of the firs 
inequality were allowed to reach 25, the v; would be telling us that another unit & 
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activity 7 = 1 would produce a net impact on the optimal value of $25 (thousand) 
yet costs only $20 (thousand). Such bargains cannot be allowed if the v; are to mean 
what we want them to mean. 


Formulate and interpret the main dual constraints of the linear programs in Sample 
Exercise ; 


Analysis: We apply principle | 7.24). 


a) For this minimize model, main dual constraints are 
t+1y, +3 —5y3 SS 
+4, +2v. —2y3 <1 
+2v, +8v2 +13 +14, <4 
+2v. +1v3 +1lv4 <5 
They may be interpreted as requiring that the net marginal value of each activity 


ender an optimal choice of dual variable values cannot exceed its given objective 
“unction cost. 


>) For this maximize model, main dual constraints are 


+1; +103 > 13 
+3 + 6v3 +1v4 > 24 
+1lyv. +8v3 B25 


+5v2 +2v3 +1v4 > 50 


They may be interpreted as requiring that the net marginal cost of each activity 
sader an optimal choice of dual variable values cannot be less than its given objective 
“sanction benefit. 


Optimal Value Equality between Primal and Dual 


~ dual variables are to price the resources associated with constraints correctly, 
= supplies and demands for those resources should evaluate to exactly the primal 
»tmal value. 


725 | 
yi oe 


all const 


In the Two Crude case, requirement 725 implies that 
20xj + 15x35 = 2vp + 1.5v3 + 0.5v3 + 9v9 + OVE 


‘Se sum of optimal prices on all constraints times current limiting values should 
"scover the optimal primal objective value. 
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EXERCISE 7.14: EXPRES ING PRIMAL-DUAL VALUE EQUALITY _ 


aye and interpret optimal value equality requirement ’ for each of the linear 
programs in Sample Exercise 7.10. 


Analysis: We want the primal optimal value to equal exactly the implicit total value 
of all constraint resources. 
(a) For this minimize model 


5x1 + Lx + 4x3 + 5x4 = 36v1 + 2502 + 7v3 + 6004 


(b) For this maximize model 


13x, + 24x. + 5x3 + 50x4 = 89v, + 60V2 + 6083 + 284 


Primal Complementary Slackness between Primal Constraints 
and Dual Variable Values 


Recall that an inequality is active if it is satisfied as an equality by a given solution 
and inactive otherwise. If an inequality is inactive at the primal optimal solution 
small changes in its right-hand side would not affect the optimal value at all; the 
constraint has slack. We can immediately deduce the value of the corresponding 
dual variable vj. 


76 | Either the primal optimal solution makes main inequality constraint i iac- 
_ tive ¢ or r the conespondy dual variable Ve = 


Equalities are always active, so there is no corresponding condition. 


Principle carries the somewhat bulky name primal complementary slack- 
ness because it asserts that either each primal inequality or its corresponding dus 
sign restriction (v; > 0 or v; < 0) will be slack (inactive) at optimality. To illustrat: 
for the Two Crude model (7.4), we first substitute the primal optimum to determines 
which constraints are active. 


+0.3(2) +0.4(3.5) =), AO) (active) 
+0.4(2) +0.2(3.5) eel. (active) 
+022) +$0.33.5) = 145 2°05 (inactive) 
+1(2) = 2 < 9 (inactive) 
+1(3.5) = 35 ent 5) (inactive) 


The last three have slack. Thus small changes in corresponding right-hand-side o» 


efficients would have no impact on optimal value. In accord with principle [7.26 
associated dual variables must = 0. 


‘SAMPLE EXERCISE 7. EMENTARY SLACKNESS 


Express and interpret primal ao leiienitaty sidelined conditions for the linear pr= 
grams of Sample Exercise 7.10. 


r—— PREPRESS 7 


7.3 Quantifying Sensitivity to Changes in LP Model Coefficients: A Dual Model 323 


(a) Primal complementary slackness conditions for this minimize model are 
+ 3x, + 2x2 + 8x34 2x4=250 or w=0 
— 5x1 — 2x2 + 1x3 4+ 1x4 =7 or v=0 
+ 1x3 + 1x4 = 60 or vw=0 


They specify that a main primal inequality’s right-hand side can affect the optimal 
value only if the constraint is active. There is no condition for equality constraint 1 
because it is always active. 


(b) Primal complementary slackness conditions for this maximize model are 
+ 1x, + 3x = 89 or v=0 
+12x3 + 5x4 = 60 or w=0 
+1 0x1, + 6%. + 8x3 4+2x4=608 or v3=0 


They, too, specify that changes in a main primal inequality’s right-hand side can 
affect the optimal value only if the constraint is active. There is no such condition 
for equality constraint 4. 


Dual Complementary Slackness between Dual Constraints 
and Primal Variable Values 


Constraints keep activity prices below true cost in minimize problems and 
above true benefit in maximize problems. At first glance it might seem reasonable 
to ask for more (i.e., demand that activity prices match exactly corresponding c;). 

There would be a practical mathematical difficulty with such a requirement in 
the common case where we have many more primal variables than main constraints. 
Relatively few v; would simultaneously have to satisfy constraints 


> Gi Min Gi 
i 


for too many activities /. 

More important, we want the dual variables to measure resource value at op- 
timality. The only (nonnegative) primal variables involved in an optimal solution 
are those with optimal x* > 0. Limiting perfect valuation to this more limited list 
of activities produces our final set of requirements—dual complementary slackness 
conditions. 


7.27 | Either a nonnegative primal variable h = 0 or the cor- 


value x, = 
responding dual prices v; must make the ith dual constraint | active. 


This time the complementarity is between the primal nonnegativity x; > 0 and the 


corresponding dual inequality | 7.24}. 
For example, the tiny Two Crude optimum had both primal variables positive 


at optimality. Principle | 7.27 | now tells us that the dual values v; we are trying to 
understand must make both constraints of (7.6) active. 
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SAMPLE EXERCISE 7.16; EXPRESSING DUAL COMPLEMENTARY SLACKNESS 


Express and interpret dual complementary slackness conditions for the linear pro- 
grams of Sample Exercise 7.10. 


Analysis: We apply principle 7.27 \to the main dual constraints of Sample Exercise 
feller 


(a) For this minimize model, dual complementary slackness conditions are 
x1=0 or 1443v2 —5v3=5 
5 or 47, +2v2 —2¥3=1 
x%3=0 or 2v,+8v.+1v3 +14 =4 
x4=O0 or 2v.+1v3+1¥4=5 
They may be interpreted to mean that marginal resource prices v; must make the 
implicit value of each activity used in an optimal solution equal its given cost. 
(b) For this maximize model, dual complemenary slackness conditions are 
x1=0 or 1y,+10v3 = 13 
x =0 or 3y,+6v3+ 14 = 24 
x3=0 or 1v2.+8v3=5 
x4=0 or 5v2+2v3+1v4 = 50 


They may be interpreted to mean that marginal resource prices v; must make the 
implicit cost of each activity used in an optimal solution equal to its given benefit. 


7 4 FORMULATING LINEAR PROGRAMMING DUALS 


The remarkable fact about linear programs is that all the quantitative sensitivity 
requirements of Section 7.3 can actually be achieved. Not only that, but they can 
be achieved with very minor computation as a by-product of the search for a prima. 
optimum. 

The secret behind these elegant results is deceptively simple: We think of dua’ 
variables v; as decision variables of a new dual linear program defined on the same 
constants as the primal and determine their values by optimizing that dual. In this 
section we show how to formulate duals, and in the next section we verify the prima! 
to-dual relationships of Section 7.3 and more. As usual, x; will always denote the jth 
primal variable, c; its objective function coefficient, and a;; its coefficient in the ith 
constraint; v; is the dual variable for the ith constraint, and b; is the right-hand side 


Form of the Dual for Nonnegative Primal Variables 


To form a dual when primal decision variables are nonnegative, we optimize tota. 


resource value )~, b;v; subject to the main dual constraints and dual variable- 
type restrictions derived in Section 7.3. — 


7.4 Formulating Linear Programming Duals 


Notice some elegant symmetries: 


A minimize primal yields a maximize dual, and vice versa. 


There is one dual variable for every primal constraint and one dual constraint for every 
primal variable. 


e Objective function coefficients of the primal become the right-hand sides of the dual, 
and right-hand sides of the primal provide the objective function coefficients of the dual. 


Our familiar Two Crude model (7.4) is a specific example. That minimize primal 
has dual 


max 2v; + 1.5v2 + 0.5v3 + 9v4 + 6v5 

st. 0.3¥1, + 0.4¥2 + 0.213 +14 < 20 
0.4v; + 0.2v2 + 0.33 + 1v5 15 (7.7) 
V1, V2, v3 = 0 
Va, V5 0 


A 


An optimal solution is vj = 20, vy = 35, v3 = vy = ve = 0, with dual objective 
function value 92.5. 


State the duals of each of the following primal linear programs. 


fa) min +30x; + 5x3 
s.t. + 1x4 —1lx. +1x%3 31 V4 
+3x, 41x =4 Dv 


+4x. +1%3 <10 V3 
X1,%2,x%3 > 0 
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(b) max +10x; 49x. —6x3 


s.t. + 2x4 + 1x2 >3 VT 
+5x, +3x. —-1x3 <15 DV 
t+1xy. +1x3 =1 V3 
X1, X2,x3 > 0 
Analysis: 


|, the dual of this minimize model is a maximize model 


(a) From construction | 7, 
with objective function 


max +1ly, +47 +1013 
derived from the RHS of ne penal: Constraints include one main constraint for each 


primal variable (principle| 7.24 |) and type restrictions from table} 7.21 | Fee. Specifically. 


st. +1y, +32 < 30 
—ly, +1 +473 < 0 
+1y, a Anes ies 5) 


vy > 0, v2 URS, v3 <0 


objective function 
min +3y, +15y. +193 
derived from the RHS of the primal. Constraints include one main constraint for each 


primal variable (principle ) and type restrictions from table| 7.21 |. Specifically. 


st. +2, +5v2 > 10 
FlViie AS¥a yee l yaya e9 
—lv. +13 >-6 

vy <0, v2 > 0, v3 URS 


Duals of LP Models with Nonpositive and Unrestricted Variables 


So far, all our duality results have assumed nonnegative primal variables. Not much 
is lost, because the majority of LP models have only nonnegative variables. Still. 
to obtain full symmetry between primal and dual, both should be allowed to have 
nonpositive, nonnegative, and unrestricted variables. 

Table 7.1 shows the complete picture. Nonpositive primal variables switch 
the direction of main dual inequalities. Unrestricted primal variables lead to dual 
equalities. 


Form ities dual of each oF the rolitwins ideal programs. 


(a) max +6x, —1x. +13x3 
s.t. +3x, +1x. +2x3 ta) 
+5x, —1x2 <6 


+1x. +1x3 >2 
xX; >0,x2 <0 


Analysis: 


7.4 Formulating Linear Programming Duals 


TABLE 7.1 Corresponding Elements of Primal and Dual 


Linear Programs 


Primal Element 


Corresponding Dual Element 


max Objective max ¥) cx; Objective min ¥; 5; v; 
form Constraint x; G;,x;2 b; Variable v;<0 
Constraint 2; a;,;x; = b; Variable v; unrestricted 
Constraint yj A,X; S b, Variable v,>0 
Variable x;20 Constraint 2; 4;,v,2 
Variable xj URS Constraint ; a,, Ma 
Variable xjS 0 Constraint ¥; ApV,S C; 
min Objective min ¥.c-x. Objective max ; by, 
form Constraint ©, a,,,x,> b; Variable v,> 0 


th geal U 


Constraint 2; 45x; = b; 


Constraint x; a jniX; <b, 
Variable x; >0 
Variable x; URS 


Variable x; 0 


Variable v, unrestricted 
Variable v,< 0 
Constraint ¥, a,,.v.< c, 


fired EY eg | 


Constraint x; Aj,V,= C; 


yt J 


Constraint ¥.a.,.v,>c, 


Met 2 ia ama 


min +7x; + 44x3 
st. —2x, —4x%. +143 
tix, +4 
+5x, -—1x2, +3xs 


x; <0,x3 >0 


IVoIA 


(a) Assigning dual variables v;, v2, v3 to the three main constraints, we apply the 
“max form” part of Table 7.1 to produce the following dual: 


min +7v; +6v2 +2y3 
st. +3, +5V2 
+1y, -1vy 4173 
+2v1 +13 


v; URS, v2 > 0,3 <0 


IV 


A 


6 
—1 
18 


Notice that unrestricted x3 yields an equality dual constraint. 


5) Assigning dual variables v1, v2, v3 to the three main constraints, we apply the 
“min form” part of Table 7.1 to produce the following dual: 


max +15y, +5v. —1193 

s.t. —2v, +1. +573 
—4y, +4 —-113 
+1y4 + 33 


vy <0, v2 > 0,v3 URS 


Za ih 
= 0 
< 44 


Again, unrestricted x2 yields an equality dual constraint. 
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Dual of the Dual Is the Primal 


With this full symmetry of Table 7.1, we can also demonstrate one final form of primal- 
to-dual symmetry. Look back at the Two Crude model dual (7.7) and suppose that 
it were the primal: 


max 2x; + 1.5x2 + 0.5x3 + 9x4 + 6x5 
st. 0.3x, + 0.4%. +0.2x3+1x4 < 20 
0.4x; + 0.2x2 + 0.3x3 + 1x5 < 15 


X1,.%2,.%3.> 0 
X4, X5 <0 


Like all linear programs, this primal must have a dual. Applying Table 7.1, we 
obtain 


min +20¥, +15v2 
st. +0.3v, +0.4v. >2 
+0.4y, +0.2v2 >1.5 
+0.2y, +03.  >0.5 
+1, 
+1v2 <6 
V4,¥2 >0 


Except for variable naming, this dual exactly matches the original primal (7.4). This 
will always be true. 


jual of the dual of any linear srocra 


SAMPLE EXE RCISE 7.19: “FORMING THE DUAL 
Show that the dual of the dual of the linear program in ‘part (a) at Sample Exercise 
7.18 is the primal. 


Analysis: Assigning dual variables w;, w2, w3 to the three main constraints of the 
dual part (a) of Sample Exercise 7.18, the dual of that dual is 
max +6w, —-Ilw2 +13w3 
s.t. +3w, +1w. +2w3 =7 
+5w, —I1w2 <6 
t+iw, +1w3; 2=2 


w, >0,w2 <0 


In accord with principle | 7.30, this is exactly the primal (with different variab’: 


names). 


7.5 Primal-to-Dual Relationships 


a. PRIMAL-TO-DUAL RELATIONSHIPS 


Having formed duals for all sorts of primal LPs in Section 7.4, we are ready to verify 
that they actually satisfy all the quantitative sensitivity requirements of Section 7.3. 


Weak Duality between Objective Values 


Dual forms 7.28 | and and Table 7.1 explicitly enforce type restrictions and 


main constraints of our Section 7.3 derivation. But what about value match oy 
Given primal solution {x;} and dual solution {v;}, a little algebraic manipulation 
shows that any objective function difference can be expressed as 


‘s CjXj = 3 biv; 
] i 
Diy CjXj (- Du Do VjQy jXj + Does va - da by; 
j ey eal i 
(x CjXj — 2 X vai) air boas VjQjjXj; — 3 bw (7.8) 
(x CjXj — s x ass) ar (x x Ai jXVVi — S bw 
i i if i 


j t 


a (« - vai) +d (x ae ) ” 


J 


~ 


ll 


Look closely at the last line of (7.8). Notice that each term in the first sum multi- 
plies slack in a main dual constraint times the corresponding primal variable value. 
Similarly, each term in the second multiplies slack in a primal constraint times the 
corresponding dual variable value. 

We can also predict the sign of all terms in that last line of expression (7.8) at 
feasible primal and dual variable values. For example, if the primal LP minimizes, 


the dual constraint corresponding to nonnegative variable x; is 
De aijVi SC; SO that slack (cj = De 4, jVi) = 0 
i i 


Thus if every x; satisfies its nonnegativity and duals v; satisfy all main dual constraints, 
every term in the first (7.8) sum is a product of nonnegative quantities. The entire 
sum would also have to be nonnegative. Similar reasoning accounts for nonpositive 
and URS variables and proves that the first sum will be nonpositive if our primal 
maximizes. 

Now consider the second sum in the last line of (7.8). It deals with primal 
constraints, which we know may be of <, =, or > form. Still, every term will be > 0 if 
the primal is a minimize and < 0 for a maximize. For example, in a minimize primal 
a < constraint 


yas < b; implies slack (x aus = 0 


J J 
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at feasible x;. But the corresponding dual variable is subject to type restriction v; < 0. 
making the (7.8) term 


(x Aj jXj — bs = 0 


at feasible primal and dual variable values. Other cases are similar. 
A uniform sign for all terms in the last line of equation (7.8) means that prima’ 
and dual feasible solution values bound each other in what is known as weak duality 


in the Two Crude model: 


20(2) + 15(3.5) = 2(20) + 1.5(35) + 0.5(0) + 9(0) + 6(0) = 92.5 (78 
Still, weak duality holds for any feasible primal and dual solutions. Suppose that 
we try x} = 9, x2 = 6 in the primal, and vy) = v2 = v3 = 20, v4 = v5 = —) in the 
dual. Simple substitution will verify that these solutions satisfy all constraints of the 
respective problems. Checking [731] , we find 
20(9) + 15(6) =270 and 2(20) + 1.5(20) + 0.5(20) + 9(—S) + 6(—5) = 35 
As required, the primal objective value is greater than or equal to the dual. 


SAMPLE EXERCISE 7.20: VERIFYING WEAK DuaLity 


For the two linear programs in Sample Exercise 7. 17 verify that the following prime 
and dual solutions are feasible, and show that they conform to weak duality propers 


a3 l). 

(a) Primal solution: x; = 1, x2 =1,%3 =1 
Dual solution: v; = 2, v2 = 6, v3 = —1 
(b) Primal solution: x; = 2, x2 = 1, x3 =0 
Dual solution: vj = —2, v2 = 4, v3 = —1 
Analysis: 


(a) To check primal feasiblity, we first note all specified x; = 0. For main prime 


constraints, 
41x; — 1x. —1x3 = +10)-10) +10) = 12 1 
+3x, + 1x2 = + Sek) = 4 
+4x> + 1x3 = +4(1) +10) =o al) 
Turning to dual feasibility, main constraints (see Sample Exercise 7.17) have 
hy PSV ee = 1) 30) = 14 <30 
—1ly; + lv. + 4v3 = —1(2) +16) +4(-]) = 0 0 


= 
a h(2 et L(= 1) =i So 


+11 ele 1v3 
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The given solution also satisfies type restrictions because 
yy=2>0 and yw=-1<0 
Objective function values for the given solutions are 


+30x1 + 5x3 = +30(1) +5) 35) 
+1v, + 4v2 +10v3 = +1(2) + 4(6) + 10(-1) = 16 


As implied by weak duality principle , the primal value is > the dual. 


(b) To check primal feasiblity, we first note again that all specified x; = 0. Verifying 
main primal constraints yields 


Be Sean Eo) = +2(2)+1(1) ee 
+5x1 + 3x2 — 1x3 = 5(2)+30) -10) = 13 <15 
+1x2 + 1x3 = +1(1) + 1(0) = al 
Turning to dual feasibility, the main constraints have 
+2v1 + 5v2 = 2(—2) + 5(4) = 16. = ahd 
+1ly; + 3v2+1v3 = +1(-2)+3(4)+1(-1) =9 > 9 
—1v2 + 13 = —1(4)+1(-1) =-5> -6 


The given solution also satisfies type restrictions because 


y=—2<0 and w=4>0 


Objective function values for the given solutions are 


+10x, + 9x2 — 6x3 = + 10(2) + 9(1) — 6(0) = 29 
+3v, + 15v2 + 1v3 = +3(—2) + 15(4) +1(-1) = 53 


For example, equation (7.9) has demonstrated strong duality for the Two Crude 
case. 


SAMPLE EXERCISE 7.21: VERIFYING ST 
-onsider the primal linear program 


max 6x; +1% 
s.t. 1x4 <i 


jel 
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(a) State the dual of this linear program. 
(b) Solve both primal and dual graphically. 
(c) Verify that their optimal values are equal. 


Analysis: Assigning dual variables v; and v2 to the two main primal constraints, the 
dual is 
min ly, +41 
st. lv, +2v>6 
+1y.2>1 


V1, V2 > 0 


(a) The following figures solve primal and dual graphically: 


dual optimal solution: 


yy 


1 2 3 4 


Optimal solutions are xf = 1, x} = 2 and vj = 4,v,=1. 


(b) Verifying the strong duality property , we have 
primal optimal value = 6xj + 1x; = 6(1) + 1) 
= 8 
1(2) + 4(1) = 1vj + 43 
dual optimal value 


ll 


Dual Optimum as a By-product 


We now wish to see why | 7.32] is true in general. For simplicity, assume that ow 
primal LP is a minimize; the argument is almost identical for a maximize. 

We proceed by looking back to the revised form of the simplex algorithm = 
Section 5.8. Recall that a key efficiency tool of the revised simplex is a pricing vector 


, obtained by solving 
VB & (cist, Cand, «++» Cmth) (71 


where Cist, Cond, »-- > Cmth Lefer to the objective function coefficients of the Ist, 2nd, . 
mth basic variables, and B is the current basis column matrix of a (primal) LP in sta 


Se ae ee ee ee ee ee ee ae 


RS EW rnin et NR AE A ih aN Retr cn ia) ng ee 
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dard form (see Section 5.1). The revised simplex algorithm always finds an optimal 
solution if the primal model has one, and it recognizes that optimality has been 
reached in a minimize problem when reduced costs satisfy 


CSG. pa a; jVi = 0 for all standard-form variables j (7.11) 


It is no accident that both the pricing vector of revised simplex and the dual 
variables of this chapter are denoted v. We will establish strong duality by showing 
that they are essentially the same thing. 


That is, we obtain an optimal solution to the dual of a linear program as a by-product 
of revised simplex solution of its primal. 

First, we want to establish that any pricing v,’s satisfying optimality conditions 
(7.11) are feasible in the dual . To do so, separate conditions (7.11) for standard- 
form variables that were decision variables in the original primal model from those 
associated with slack variables added to achieve standard form. For decision vari- 
ables, c; and all a;;’s come directly from the given primal. Then by (7.11), pricing v;’s 


satisfy 
Cj — a) aijVi = 0 or Ss QijVi SC; 
i i 


This is exactly what is needed for feasibility in all main constraints of dual [7.28]. 
Now focus on any slacks. We constructed standard form by choosing Ga for 
slack variable x; in primal inequality i, and 
+1 ifk =i andi is of < form 
ajA24—-1 ifk =iandiis of > form 
0 otherwise 


Thus, with each slack column having only one nonzero entry, optimality test Ga) 
on a slack in a < constraint amounts to 


(+1)v; = 0 
For a > we have 
(-1)v; <0 or vy; <0 


We conclude that pricing v;’s also satisfy the type restriction constraints in dual 2). 
They are completely dual feasible. 

Our next task is to show that the dual solution obtained from revised simplex 
pricing has the same objective function value as the primal optimum. The dual 
»bjective value for those v;’s is [using construction (7.10)] 

iy divi = sy vibj = [ (ers C2nd>---; min) B | -b 
i i 
Here b denotes the vector of b;. But we also know that the only nonzero components 
of the primal optimal solution x* are the basic ones, 


-1 
(Xist, X2nd,+--.Xmth) = Bb 
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Thus the primal optimal value 


a 
> CjXj = CistX1st + CondXand + - +. + Cmth¥mth 
J 


(Cist: Conds +++; Cmth) , (B 'b) 
= [ist Cond, -.-,mtn)B71] +b 
That is, both primal and dual objective have the same value. 

We already know that the primal solution x* is optimal in its problem; revise: 
simplex stopped. To see that pricing v;’s are also optimal in the dual, we exploit weal 
duality . Weak duality holds for every primal feasible solution, so choose x” 
No dual feasible solution can ever have a better dual objective function value tha: 
corresponding primal cost yi cjx;. But we already knowa dual feasible solution (ow 
pricing v;’s) that can achieve that limit. We conclude that those v;’s form a feasibl. 
solution in the dual having the best possible objective function value, precisely wha 
it means to be dual optimal. 


The LP 
max +5x,; +7x, +10x3 
s.t. +2x, —1x, +5x3 < 10 
+1x, +3x2 <al 5 
X1,X2,X%3 > 0 
was converted to standard form by adding slack variables x4 and xs in main constrain® 


(principle ). Then, revised simplex Algorithm 5C terminated with optimal basi 
matrix 


Xo, art) way ag) 
B= (5 —1) havinginverse B'= (2% 


(a) State the dual of the original linear program. 
(b) Use B~! to compute pricing vector v and verify that it is feasible in the dual. 


(c) Use B! to compute the primal optimal solution, and verify that it has the sam: 
objective value as the primal. 


Analysis: 
(a) Following construction , the dual is 
min +10y, +152 
st. +2y, +1w > 5 
-ly +3 2 7 
+5y1 = 
V1, ¥2 20 
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(b) Here the first basic variable is x3 and the second is x2. Thus equation (7.10) gives 


raha Jieily 
V = (Cist, Cona)B-' = (10, »( - » i. 2,3) 


aly 


To check dual feasibility of v, first note that both components are nonnegative as 
required by type restrictions. In main constraints, 


Qyiret dy Sae2QDiesiG@eai heb 5 
—ly,4+3¥y. = -—12)4+33) =7 > 7 
51 = yO) = 1010 


(c) The primal optimal basic solution has nonbasics x} = x; = xg = 0 and basic 


components 
= sanetill + LLC go 
ap 0 15 e 


Thus the primal optimal value is 5(7) + 10(3) = 65, which is the same as dual’s 
10(2) + 15(3). 


ale ale 


Complementary Slackness between Primal and Dual Optima 


Complementary slackness conditions and also hold when both primal 


and dual have an optimal solution (principle |7.32]) as a consequence of strong 
duality. To see why, observe that one way to express dual complementarity conditions 


7.27 | is 


(dual slack) (primal variable); = (« — is, a) xj = 0 (7.12) 
i 


Either slack or variable must be = 0, possibly both. Similarly, primal complemen- 


tarity conditions amount to 


(primal slack),(dual variable), = (x Gi jXj — ) vi =0 (7.13) 
j 

Now look back to weak duality computation (7.8). There we expressed the 

difference between primal and dual objective function values as sums of terms exactly 

ike (7.12) and (7.13). In our derivation of weak duality we also concluded that 

cach such term would be nonnegative in a minimize problem and nonpositive in a 

maximize problem. The only way that such terms of uniform sign can sum to the zero 

difference we know follows from strong duality is for every term to be zero. That is, 
strong duality’s 


0= Sox -Eibot 
ih i 
= ¥(4-¥ vies) t+ (Naat -0) 
J i i j 
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implies that optimal primal and dual solutions satisfy every complementary slackness 
condition (7.12)-(7.13). 


(a) Write all primal and dual complementary slackness conditions for model (7.12)- 
(7.13) in product form. 


(b) Verify that the conditions are satisfied by optimal primal and dual solutions 
computed in Sample Exercise 7.22. 


Analysis: 
(a) Primal complementary slackness relates primal inequality slack to dua’ 
variable values: 
(2x, — Lxp + 5x3 —10)v, = 0 
(1x, + 3x2 — 15)v2 =0 
Dual complementary slackness relates dual inequality slack to primal variable 
values: 
(5 = 2V1 = 1v2)x1 =0 
(7 +11 — 3v2)x2 = 0 
(10 — 5v1)x3 = 0) 


(b) From Sample Exercise 7.22, x* = (0,5, 3) and v* = (2,3). Checking prima 
complementary slackness conditions, we find that 
(2x1 — 1x2 + 5x3 —10)v1 = [2(0) — 1(5) + 5(3) — 10](2) =0 
(Lay + 3x2 — 15)v2 = [1.(0).+:3()— 15]@) =0 
Similarly, in dual complementary slackness conditions, 
(5 —2vy — 1v2)x, = [5 — 2(2) —10)]@) =0 
(7 + 1v1 — 3v2)x2 = [7+ 1(2) — 3(3)]6) =0 
(10 — 5v)x3 = [10 — 5(2)](3) = 0 


Unbounded and Infeasible Cases 


Strong duality and complementary slackness conditions depend on the primal (ane 
thus the dual) having optimal solutions. We would hardly undertake post-optimalin 
analysis if there were no optimal solution. Still, other cases are sometimes of interest 

We learned very early that LPs can prove infeasible or unbounded. Weas 


duality condition | provides the key tool in infeasible and unbounded instances 
Suppose, for example, that a primal minimize problem is unbounded. Weak dualit 

says that any dual feasible solution yields a lower bound on primal objective functios 
values. But if the primal is infeasible, no such bound exists. The only possib 
explanation is that no dual feasible solutions exist. 
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We could make a similar argument for unbounded duals. Any primal feasible 
solution would limit dual objective function values, so none can exist. 


7.34 | Ifeithera LP model or its dual is unbounded, the oe is a 
sible. Se 7 


After seeing so much elegant symmetry between primal and dual, one might 
guess that infeasibility in one problem also implies unboundedness in the other. This 
is false! Both primal and dual in the following are obviously infeasible: 


min —x; max vj; 
(primal) st. 2x,-x2 > 1 (dual) st. vw+y < —-1 
Xy—X. < O —vWy-v2 < 0 
X1,X%2 > 0 vy, >0, v2 <0 


Dual 


Primal Optimal_—Infeasible + Unbounded — 
_ Optimal Possible Never —--- Never. 
- Tnfeasible Never Possible Possible 


-... Unbounded... Never _ Possible Never. = ee 


(a) Show Desreden that the following li linear program is unbounded and verify the 
implication for its dual. 


max +1x; 
s.t. —1x, +1x >1 
+ 1x, >2 
X1,X%2 > 0 


(b) Show graphically that the following linear program has an unbounded dual and 
verify the implication for the primal. 

min +1x, +1x) 

Sub es Stage elton 23 


+5x, +1xy <0 


Same gee 0) 


Analysis: 
a) The following plot clearly shows that the model is unbounded: 
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dos nda 
Its dual is 
min +1), 
s.t. —1y >1 
+1y, +1, >0 
V1, V2 <0 


which is infeasible as implied by principle 5 


(b) The dual of this linear program is 
max +37 
st. +2, +5. <1 
+1y, +1 <1 
vy >0,v2 <0 


Solving graphically gives 


which is clearly unbounded. It follows from principle that the primal 
infeasible. 


) & COMPUTER OUTPUTS AND WHAT IF CHANGES 
OF SINGLE PARAMETERS 


All the machinery for quantitative sensitivity analysis is now in place. It is time» 
do some. 
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CFPL Example Primal and Dual 


We illustrate with the CFPL model formulated in Section 4.3 to plan operations of a 
plywood factory. Table 7.2 displays the full primal in terms of the decision variables 


Wq,v,. 2 number of logs of quality g bought from vendor 
v and peeled into green veneer of thickness ¢ per 
month 


Xt, 4 number of square feet of grade g green veneer of 
thickness ¢, purchased directly per month 


Yig.g° & number of sheets of thickness ¢ used as grade g/ 
veneer after drying and processing from grade g 
green veneer per month 


Zi.g,¢' 4 number of sheets of front veneer grade g and back 
veneer grade g’ plywood of thickness ft, pressed and 
sold per month 


A dual variable v; has been assigned at the extreme right of each constraint. The 
optimal solution value is $484,879 per month. 
Table 7.3 provides the corresponding dual. Key points in its derivation include: 


e The dual minimizes because the primal maximizes. Objective function coefficients come 
directly from the primal right-hand side. 

e There is one main constraint for each of the 32 primal variables. All such constraints 
have the > form because all primal variables are nonnegative. Dual right-hand sides 
come from the objective function coefficients of corresponding primal variables. Dual 
variables weighted on the left-hand side are those for constraints where the associated 
primal variable has nonzero coefficients. 

e The first 17 dual variables are nonnegative because they correspond to < constraints in 
a maximize primal. The next 6 are nonpositive because they relate to >’s, and the last 
5 are unrestricted because they correspond to =’s. 


Constraint Sensitivity Outputs 


No two linear programming codes are identical, but all report the primal optimal 
solution, the corresponding dual solution and some related sensitivity information. 

Table 7.4 displays a typical output of constraint information. For each of the 28 CFPL 
primal) constraints, the printout shows 


Typ Whether the constraint is of L= <, or G=2, or 
E = equality form 
Optimal Dual The optimal value of the dual variable for the 


constraint 
RHS Coef The specified right-hand side for the constraint 
Slack The amount of slack in the constraint at primal 
optimality 
Lower Range The lowest right-hand-side value for which the 


optimal dual solution must remain unchanged 
Upper Range The highest right-hand-side value for which the 
optimal dual solution must remain unchanged 
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TABLE 7.2. CFPL Example Primal 


maximize: 

(log costs) 

-340WG, 1, 116-190 pe 1, 116 — 490, 2, 116 — 140 pe 2, 1716 

—340WG, 1,11 - 190Wp 1, 1/8 - 490WG, 2, 1/8 - 140W pr, 2. 178 

(green veneer costs) 

AGtteiys ¢—- O50 ns gO 10H ng e — 2 Mitge, «= One =O 20e es 
(finished plywood sales) 

+4521/4, a, B+ 40214, 4, c + 33214, B, c+ 792172, 4, B + 952172, a, c+ 502172, B,C 


subject to: 

(log availability) 

WG.1,116 + %G,1, 118 < 200 Y4 
We. 1,116 + We, 1, 1/8 < 300 Vo 
We, 2, 1/16 + WG, 2, 1/8 < 100 V3 
(purchased green veneer availability) 

Wr, 2,116 + We, 2, 1/8 < 1000 V4 
mee < 5000 Vs 
x /16, B < 25,000 V6 
X1/16, Cc Ss 40,000 Va 
X18, A <= 10,000 Vg 
X18 B < 40,000 Vo 
x; ae s 50,000 Vio 
(plywood market limits) 

21/4, A,B < 1000 ° V4 
21/4, A,C < 4000 Vi9 
21/4, B,C < 8000 V43 
21/2, A,B < 1000 Vi4 
ve A.C <= 5000 V45 
21/2, B,C < 8000 V16 
(plywood pressing capacity) 

0.252 4)4, A,B ae 0.25244, Ke ste 0.2524 /4, B,C ar 0.4024), A,B + 0.4021), 4, Gak 0.4024), Bic < 4500 Viz 
(green veneer balance) 

400W6. 1,116 + 200Wp 1, 116 + 400WG 2, 116 + 200Wp, 2, 16 + * 116,.A 

~35y 116, A,A~ 35116, A, B 20 Vis 

700We, 1,116 + 500WF, 1, 116 + 700WG, 9, 116 + 500M 2, 1716 + ¥ 1/16, B 

~35y1n6,B,A— 35Yin6, B,B + 35116, B,C 20 Vis 
900We. 1,116 + 1300We 1, 1716 + 900WG_ 2, 116 + 1300W pe 9 1716 + ¥ 1/16, C 

-35yin6, cB - 35y116,C,¢ 20 Vag 

200WG,1, 1/8 + LOOWE 1, 1g + 200WG 9, 1/9 + 100We 9 1/9 +X 1/94 

-35y1g, 4,4 — 3518, A, B 20 V4 

350WG, 1,118 + 250Wp, 1, 178 + 350WG, 2, 118 + 250Wp 2, 118 + ¥ 118, B 

-35y1/3, 8, A — 35Y 18, B, B- 35Y 18, B,C 20 Yoo 

450WG, 1,18 + ©50Wp 1, 1/8 + 450WG, 2, 1/8 + 650Wp 2 118 + X18, C 

—35y 18, c,8 ~ 35918, c,¢ 20 V93 
(finished veneer sheet balance) 

Vine, A,A + Vie, B,A~ 21/4, A, B~ 21/4, A, C7 21/2, A, B ~ 21/2, A, C =0 Vos 

Y1i6, A,B * Y1/16, B, B + 21/16, C, B~ 21/4, A, B ~ £1/4, B, C~ £1/2, A, B ~*1/2, B, C =0 VE 
Vine, B,C * Yin6, c, Cc 21/4, A, C~ 2114, B, © 212, A, C~ 2112, B,C =0 V25 
Y18, A,B + 1/8, B, B + 21/8, C, B ~ 21/2, A, B~ 2112, A, C7 21/2, B,C =0 Va 
Yus,B,Ct Yug,c,c~ 21/4, A,B ~ 214, A,C~ 2114, B,C + 22172, 4,8 + 2Z12,4,C+ 2212,3,c = 2 
(nonnegativity) 


all variables = 0 
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TABLE 7.3. CFPL Example Dual 


minimize: 
200v, + 3002 + 100v3 + 1000v4 + 5000v5 + 25,000V, 


+ 40,000v7 + 10,000v¢ + 40,0009 + 50,000149 + 1000v,, + 4000r,5 
+ 8000¥43 + 1000¥44 + S000v,5 + 8000115 + 450017 


subject to: 
(w-variable columns) 


Vz + 400V1g + 700V19 + 900V 29 2-340 
Vo + 200V4g + S00V49 + 1300V29 2-190 
v3 + 40018 + 70019 + 900V29 2-490 
V4 + 200V1g + 500V49 + 1300V9 2-140 
V1 + 200V5; + 350v29 + 450123 2-340 
Vz + 10021 + 250199 + 650V53 2-190 
¥3 + 200V21 + 350V29 + 450V95 2-490 
¥4 + 100v21 + 250v9 + 650V93 2-140 


(x-variable columns) 


V5 +Vig > -1.00 
V6 + Vi9 >-0.30 
V7 + Voq 2-0.10 
Vg + V4 22.2 
Vo + ¥99 >-0.60 
Vig + V3 >-0.20 
(y-variable columns) 

V24 — 3548 20 

vo4 - 3549 20 

V5 — 3548 20 
Vos 35V49 20 
V95— 35V29 >0 

V6 - 35V 49 20 

96 - 35 V9 20 

V7 — 3591 20 
Von 35¥99 20 
va7— 35¥23 >0 

Vg — 35 V9 20 

Vg — 3523 20 
(z-variable columns) 

V4z + 0.25017 — V4 — Vo5 — Vg 245 
V9 + 0.25047 — V24 — V26 — Vg 240 
V43 + 0.25v17 — Vo5 — Vog — V2g 233 


V44 + 0.40¥47 — V94 — V95 — V97 — 2v9g > 75 
5 + 0.40047 — Vag — V26 — Vo7 — 2V2g 2 65 
V46 + 0.40047 — ¥25 — V26 — Vo7 — 2Vng > 50 


= 


(sign restrictions) 


V1, V29+++,¥y720 
Vig, Vj9)+++5¥23 $0 
V4, V25,---, Vag unrestricted 


Notice that slack and dual values conform to primal complementary slackness 


sonditions | 7.26]. Dual variables are positive only when the corresponding slack is 
BETO. 
Right-Hand-Side Ranges 


The last two items for each constraint are new to our discussion. To understand 
eir meaning, look back at Figure 7.3. We know that optimal dual variable values 
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TABLE7.4 Typical Constraint Sensitivity Analysis Output for CFPL Model 


Name Typ Optimal RHS Slack Lower Upper 

Dual Coef Range Range 
él L -0.000 200.000 158.727 41.273 +infig 
c2 L U22. 156 300.000 0.000 242.500 33: oR 
@3 L -0.000 100.000 100.000 0.000 +infig 
c4 L LF2< 1516 1000.000 0.000 942.500 103.199 
aS L -0.000 5000.000 5000.000 0.000 +infiz 
cé L -0.000 25000.000 25000.000 0.000 +inftig 
ey. L 0.032 40000.000 0.000 0.000 85858 .586 
c8 L -0.000 10000.000 10000.000 0.000 +intig 
9 L -0.000 40000.000 40000.000 0.000 +infis 
c10 L OL 12 50000.000 0.000 0.000 81971.83 
edu L 9.564 1000.000 0.000 342.857 2621,42 
eae L 4.564 4000.000 0.000 3342.857 5621.42 
e13 L -0.000 8000.000 3644.156 4355.844 t+infin 
el4 L 10.000 1000.000 0.000 402.597 5402.597 
rol) L -0.000 5000.000 597.403 4402 .597 +infi 
c16 L -0.000 8000.000 8000.000 0.000 +infa 
ely L 51.418 4500.000 0.000 4404.622 4609 .525 
c18 G =O0.202 0.000 -0.000 Soult ee Al 131759.46 
e19 G =05:2/011 0.000 -0.000 Soll eal 194861.11 
c20 G =0:-13.2 0.000 -0.000 -45858.586 41818.16& 
era: G = Osae2 0.000 -0.000 =31975..83 57500.000 
e22 G =Oresi 2 0.000 -0.000 SS 197831 5/500.00 
ens) G =07.312 0.000 ~ -0.000 SOLO 7s 83 57500 .00 
c24 E -7.046 0.000 0.000 -1460.317 5567.46 
@25 E -4.610 0.000 0.000 -1310,245 1194. 805 
c26 E -4.610 0.000 0.000 -1310.245 1194.80 
oval E =NON7925 0.000 0.000 -913.48 1642.85" 
c28 E SO 9iZ5 0.000 0.000 -913.48 1642.85" 


quantify the slope or rate of change in such relations between the right-hand side 
of a constraint and the overall optimal value. In the CFPL output, for instance. 
v3 = 122.156 implies that the optimal value will improve by $122.156 per log increase 
in RHS = 300 of the corresponding log availability constraint. 

The RHS range question is how much we could change the right-hand sid= 
before that ps would no longer apply. 


which the corresponding he variable alue ae the exact rate Sof oes 
in optimal value per unit change it in RHS : fall other data held constant). 


For example, output Table 7.4 tells us that slope v3 would remain unchanged for any 
RHS in the interval [242.5, 331.972]. Outside the range, we have only qualitative 
principle : 

The idea behind range computation is to identify the maximum change in any 
RHS for which the optimal basis remains (primal) feasible. In rows with slack, such as 
the first constraint in Table 7.4, feasibility persists exactly until the slack is eliminated 
Thus with the original constraint one RHS of 200 permitting 158.727 units of slacks 
dual slope vj = 0 applies for any RHS value at least 


RHS — slack = 200 — 158.727 = 41.273 
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The upper range is +oo because increasing the RHS only relaxes an already slack 
constraint. 

The computation is a bit more complex when a constraint is active, but the issue 
remains keeping the primal solution feasible. We omit details. 


Suppose that solution of the LP 
min +5x,; +9x2 
st. +1x, +1% >=3 
+1x, -ly <4 
x1, % = 0 


produces the constraint sensitivity output 


Name Typ Optimal RHS Slack Lower Upper 

Dual Coef Range Range 
a G 510,00 3.000 -0.000 0.000 4.000 
2 L 0.000 4.000 1.000 3.000 Timea 


Sketch what can be deduced from this output about how the optimal value varies 
with each RHS. 


Analysis: What we can know about how optimal value varies with each RHS is 
summarized in the following two plots: 


optimal optimal 
value value 
A A 


I | 
[aot RH 
3.0 4.0 ao 


Constraint 1 Constraint 2 


slope 5.0 slope 0.0 
j ee 
| 
| 


The printout’s RHS range for constraint 1 shows that the rate of change in optimal 
alue with its right-hand side is dual variable value 5.0 in the range 0.0 to 4.0 (principle 
7.36] ). Above that range qualitative principle implies that the rate can only 
acrease because we are tightening a > constraint. Below the range the rate-can only 
decrease. 
For constraint 2 the RHS range is [3.0, +00). Within this range there will be no 
change in optimal value because the corresponding dual variable is 0.0. Below the 


range the tightening < may produce a steeper rate of change (principle y 


343 


344 


Chapter Seven e Duality and Sensitivity in Linear Programming 


Constraint What If’s 


There are hundreds of possible “what if” questions about the CFPL model that can 
be asked and answered with the aid of results in Table 7.4. Much can be said if 
we consider changing only one coefficient while holding all others constant. We 
illustrate with a few examples. 


e Question: How sensitive is the optimal value to our 8000 estimate of the market for 
;-inch BC plywood? 
Analysis: The market constraint for $-inch BC has optimal dual v*, = 0 and 
RHS range [0, +00). Thus the optimal value will remain unchanged no matter what 
(nonnegative) market estimate we employ. 


e Question: How sensitive is the optimal value to our 300 estimate of the supply of “fair™ 
logs available from supplier 1? 

Analysis: The availability constraint for fair logs from supplier 1 has optimal dual 
vj = 122.156 and the RHS range [242.5, 331.972]. Thus, if the estimate of 300 is toc 
high, every log deducted will reduce the optimal value by at least $122.156. If it is toc 
low, every log added will increase the optimal value by at most the same amount. 


optimal slope 
value 122.156 


| 

| 

| 

| 

I 

| 

| 

! 
NN 
BR 
oO 
= 
ise) 
on 


The $122.156 rate is exact within the RHS range. Below the range, the rate of change 


will become steeper because we are tightening a constraint (principle | 7.14 |). Above 
the range, the rate will diminish. 


e Question: What is the marginal value of plywood pressing capacity? 
Analysis: The marginal value is the optimal dual on the pressing capacity con- 
straint, vj, = $51.418 per sheet. 
e Question: How much should we be willing to spend in promotional costs to increase 
the market for ;-inch AB plywood from 1000 to 2000 sheets per month? 
Analysis: Because 2000 is within the RHS range on the market constraint for 


;-inch AB, dual vj, =$9.564 gives the exact rate of change. An increase from 1000 t 
2000 would be worth 


(new value — current RHS) vj, = (2000 — 1000) (9.564) 
= $9564 per month 


Any promotional expense up to that amount would be justified. 


e Question: How much should we be willing to pay to increase plant pressing capacity 
from 4500 to 6000 sheets per month? 
Analysis: New value 6000 falls well beyond the upper range limit of 4609.624 o= 
the pressing capacity RHS, thus we can only bound the value of the proposed capacity 
increase. 
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value bound 


| 
| 
I 
| 
| 
| 
| 
| 
| 
a 
= 
S 
S 
=) 


4404.622|--~-- 
4609.524, ------ 


At a minimum, we know that the optimal value would improve: 
(range limit — current RHS) vj, = (4609.624 — 4500) (51.418) 
~ $5637 per month 


The dual provides an exact rate of change through the upper range limit. 
Above the range limit, the rate of change may decline (principle 


are relaxing a constraint. Still, the gain from raising capacity to 6000 cannot exceed 
the estimate provided by the dual-variable value: 


(new value — current RHS) vj, = (6000 — 4500) (51.418) 


= $77,127 per month 


We conclude that any equivalent monthly capacity expansion cost up to $5637 could 
be justified and any over $77,127 could not. In between, we cannot be definitive. 


e Question: How much loss would we incur if we stopped using “good” logs from supplier 
2? 
Analysis: The dual variable on supplier 2 good log availability has v3 = 0.0 and 
RHS range [0.0, +00). Thus no change in optimal value would result if we reduced 
availability all the way to zero. 


e Question: How much loss would we experience if we stopped using fair logs from 
supplier 2? 

Analysis: The dual variable on supplier 2 fair log availability 1000 has vj = 172.156 
and RHS range [942.5, 1031.972]. Eliminating such log purchases amount to changing 
the RHS to zero. For such a value, which is well outside the RHS range, we can only 
bound the effect. 


slope 


timal 
oP. Saas 


value 


bound) y 


- 


i) 
i] 
=) 
boa 


fan} N 
> > 
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We are tightening the availability constraint, so principle shows that the 
true rate of change will be vj (actually, its negative) or worse. Thus, closing out 
supplier 2 fair logs would produce a loss of at least 


(new value — current RHS) v3 = (0 — 1000)(172.156) 
= — $172,156 per month 


There is no way to use Table 7.4 information to set an outer limit on the loss (except 
the full optimal value, $484,878). 


Variable Sensitivity Outputs 


We saw in Section 7.1 that sensitivity analysis encompasses much more than just 
changes in right-hand-side values. Table 7.5 provides an example of the variable 
or column-oriented sensitivity information that is part of almost any LP optimizer 
output. The following are displayed for each primal variable: 


Name The name of the primal variable 
Optimal Value The optimal value for the variable 
Bas Sts Whether the variable is BAS = basic, NBL 


= nonbasic lower-bounded, or NBU = nonbasic 
upper-bounded in the optimal solution (see 


Section 5.9 
Lower Bound The specified lower bound on the variable 
Upper Bound The specified upper bound on the variable 
Object Coef The specified objective function coefficient for 
the variable 


Reduced Object The reduced objective function coefficient for 
the variable at optimality 

Lower Range The lowest objective function coefficient 
value for which the optimal primal solution 
must remain unchanged 

Upper Range The highest objective function coefficient 
value for which the optimal primal solution 
must remain unchanged 


Most of these items are familiar from our previous discussion. Optimal values 
objective function coefficients, and reduced objective function coefficients have usu- 
ally been denoted x*, cj, and ¢;, respectively. Notice that given values satisfy dua! 


complementary slackness conditions | . Whenever Optimal Value> 0, the 
corresponding dual constraint slack Reduced Object =0. 


Objective Coefficient Ranges 


As with constraint information, the two completely new items are ranges, this time om 
the objective. Items Lower RangeandUpper Range delimit changes in objective 
coefficient value that would leave the primal optimal solution unchanged. 

Figure 7.5 illustrates how keeping the same optimal solution does not neces- 
sarily mean keeping the same optimal value. The optimal level for sales variable 
21/4,4,8 (2GAB in the computer output) is 1000. Thus the optimal objective value 
will increase at the rate of $1000 per $1 increase in the z1/4,4.5 objective function 
coefficient as long as the primal optimal solution remains unchanged. 
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TABLE 7.5 Typical Variable Sensitivity Analysis Output for CFPL Model 


Name Optimal Bas Lower Upper Object Reduced Lower Upper 
Value Sts Bound Bound Coef Object Range Range 
wGls 41.273 BAS 0.000 +infin -340.000 OOOO. =3.61'. 569°" =304. 762 
wFls 300.000 BAS On O00) = shiinitainy 1910-1000, 0.000" =190 ..000 t+infin 
wG2s 0.000 NBL 0.000 +infin -490.000 150.000 -infin -340.000 
wE2s 552273 BAS 0.000 +infin -140.000 0.000 -185.306 -140.000 
wGle 0.000 NBL 0.000 +infin -340.000 27.844 Sims | oles 56 
wFle 0.000 NBL 0.000 - tintin —-=190..000 0.000 =infin <-190.000 
wG2e 0.000 NBL 0.000 +infin -490.000 177.844 Sinead 322). 156 
wi 2e 844.727 BAS 0.000 +infin -140.000 0.000 -140.000 -94.694 
xsA 0.000 NBL 0.000 +infin -1.000 0.799 -infin =0.201 
xsB 0.000 NBL 0.000 +infin -0.300 0.099 -infin -0.201 
xsC 40000.000 BAS 0.000 +infin -0.100 0.000 =aOiaiar2 +infin 
xeA 0.000 NBL 0.000 +infin 522.00) 1.888 =indéam Osa? 
xeB 0.000 NBL 0.000. =tiintan -0.600 0.288 ahi an 0/312 
xeC 50000.000 BAS 0.000 +infin -0.200 0.000 Oe sl +infin 
zqAB 1000.000 BAS 0.000 +infin 45.000 0.000 35.436 ant ath 
zqAC 4000.000 BAS 0.000 +infin 40.000 0.000 35.436 +infin 
zqBC 4355 .844 BAS 0.000 +infin 33.000 0.000 34-612, 34.675 
zhAB 1000.000 BAS 0.000 +infin 75.000 0.000 65.000 t+infin 
zhAC 4402 .597 BAS 0.000 +infin 65.000 0.000 62.320 67.220 
zhBC 0.000 NBL 0.000  tinfin 50.000 12.564 -infin 62.564 
yeBB 0.000 NBL 0 4000" “stinkin -0.000 0.000 cibrala me kia 0.000 
ysAA 3073.247 BAS 0.000 +infin 0.000 0.000 a2 3s. 5.045 
ysAB 0.000 NBL 0.000 +infin 0.000 2.436 -infin 2.436 
ysBA F329). 354: BAS O4000 . ) shamBan 0.000 0.000 =2. 792 3.964 
ysBB 0.000 NBL 0.000 +infin 0.000 2.436 Sane in) 2.436 
ysBC 0.000 NBL 0 2000 2 tintin 0.000 2.436 -infin 2.436 
ysCB 6355.844 BAS 0.000 +infin 0.000 0.000 =1. 8:38 1.675 
ysCc 12758.442 BAS 0.000 +infin 0.000 0.000 a3 00) 4.467 
yeAB 2413.506 BAS 0.000 +infin 0.000 0.000 P1287 15}; 85i7 
optimal 
value 
4 slope 1000.000 sp 

| 

| 

| 

\ | 

i | 

\ | 

| | 

Ri PS __» obj 
35.436 45 


FIGURE 7.5 Optimal Value Impact of CFPL 
Objective Changes 
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=: “obj ective coefficients (all other ata held constant), 


In Figure 7.5 the range is [35.436, +00). Outside the range we must rely on 
qualitative principle to impute rates of change. Since selling prices below 
$35.436 help the objective less, the rate < 1000. 

Like the RHS case, objective coefficient range computations involve details 
inappropriate for this book. However, calculations center on keeping the dual solu- 
tion feasible so that optimality conditions will continue to hold for the current primal 
optimum. 

The computation is easy when dual slack ¢; is nonzero. Certainly, dual feasibility 
will not be lost until all slack is eliminated. For instance, sales variable z, /2,B,C (zhBG), 
with reduced objective coefficient —$12.564 on an original coefficient of $50, has 
objective coefficient range (—o0o, 62.564]. Reductions from $50 only relax an already 
slack dual constraint, and increases eliminate slack only at the coefficient value 


(current coefficient — dual slack) = 50 — (—12.564) 
= $62.564 per month 


Return to the LP of Sample Exercise 7.25, and. assume ‘that ihe variable part of 
sensitivity output is as follows: 


Name Optimal Bas Object Reduced Lower Upper 
Value Sts Coef Object Range Range 

ell 5.000 BAS 5.000 0.000 0.000 9.000 

po 0.000 NBL 9.000 4.000 Spe OOO | Srtanait asa 


Sketch what can be deduced from this output about how the optimal value will change 
with the two objective function coefficients. 


Analysis: What we can know about how the optimal value varies with objective 
function coefficients is summarized in the following two plots: 


optimal optimal 
value value 
A 
slope 0.000 ~ 
oe | 
| I 
| } 
I | 
- . - D obj 
5.000 9 
Variable 1 Variable 2 


The range on the first coefficient is [0.0, 9.0], meaning that the current primal solution 


| 


7.6 Computer Outputs and What If Changes of Single Parameters 


will remain optimal for any c; in the range and that the optimal value varies within 
that range at rate x* = 3.0 (principle | 7.38). Increasing that coefficient hurts the 
optimal value, so the rate may decline for cy > 9.0 (principle [7.16]). Below the 
range the rate becomes steeper. 

Similarly, the output above shows that the range for c2 is [5.0, +00). Any cost in 
that range will leave x3 = 0.0 and the optimal value unchanged. Below the range the 
objective may decline because helping the objective will steepen the rate (principle 


y). 


Variable What If’s 


The value of output information in Table 7.5 is best illustrated by considering some 
of the “what if” questions that it might help to answer. Again we assume only one 
parameter changes with all else held constant. 


e Question: How sensitive are the optimal solution and value to our $33 estimate of the 
selling price for +-inch BC plywood? 
Analysis: The objective range of [31.612, 34.675] on zqBC implies that the optimal 
operating plan would remain unchanged for any price between those values. However, 
the optimal value would vary. 


optimal 
value 
slope 4355.844 


obj 


| 
| 
I 
1 
on 
lon) 


34.675 


Every price increase of $1 would add z}/, 3 ¢ = $4355.844 per week to the optimal value 


within the range and that much or more above (principle ). Every $1 decrease 
would reduce the optimal value at the same rate through limit $31.612 and that much 
or less thereafter. 

e Question: How sensitive are the optimal solution and value to our $1.00 per square foot 
(objective coefficient -1.00) estimate of the cost of ig-inch A green veneer? 

Analysis: Coefficient range (—co, —0.201] on xsA implies that the optimal solu- 
tion would remain unchanged for any coefficient below —0.201 (cost above $0.201) per 
square foot. Within that range, optimal value would also remain unchanged because 
X16. = 0- 

optimal 
value 


slope 0.000 


—> obj 


| 

| 
= 
So 
St 
t 
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Coefficient values above —0.201 (costs below $0.201) might improve the optimal value 
because the rate of change cannot decrease. 
e Question: How much difference in the optimal value would it make if we increased or 
decreased the $65 per sheet selling price of 5-inch AC plywood by 20%? 
Analysis: A 20% increase would bring the price to (1.2)(65) = $78. Because this 
value is well beyond the upper range limit of $67.220 for variable zhAC, we can only 
bound the optimal value impact. It will equal or exceed 


(new coefficient — original coefficient) Zina = (78 —65)(4402.597) 
x $57, 234 per month 
optimal 
value 


67.220} ---- 


A 20% price decrease also takes us well outside the range. We can compute the 


least possible impact by considering only the range where we know the exact rate of 
change. 


(range limit — original coefficient) Z12,4,c = (62.320 — 65)(4402.597) 
«x — $11,799 per month 


Extending the current rate of change all the way to (0.8)(65) = $52 bounds the maximum 
impact: 


(new coefficient — original coefficient) zi, 4 ¢ = (52 — 65)(4402.597) 
= — $57,234 per month 


e Question: The current optimal plan produces no $-inch BC plywood at our estimated 
$50 per sheet selling price. At what price would producing that product become prof- 
itable? 

Analysis: Coefficient ranges indicate z* 2,.B,c = 9 would remain the optimal choice 
for any price up to $62.564. At prices above that value a positive profit is possible because 
the rate of change can only increase as we move a coefficient to help the objective 


([746]). 


e Question: The current optimal plan buys x¥ ji6,c = 40, 000 square feet of #-inch C green 


veneer per weck at cost $0.10 (objective coefficient —0.10) per square foot. At what 
price would such purchases become unprofitable? 

Analysis: The coefficient range for X1y16,c 18 [—0.132, +00). Thus we know that 
the 40,000-square foot value will remain optimal at coefficients above —0.132 (i.e., cost 
below $0,132 per square foot). Beyond $0.132 it may be optimal to buy less. 


Dropping and Adding Constraint What If’s 


Another familiar form of “what if” questions involve dropping or adding constraints. 


Results in Figures 7.4 and 7.5 help here, too. The necessary insights are almost 
obvious: 
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Some examples (assuming the rest of the model is held constant): 


e Question: Would dropping the press capacity constraint change the optimal plan? 
Analysis: Yes. The constraint has no slack at the optimal solution. 
e Question: Would dropping the supplier 1 good log availability constraint change the 
optimal plan? 
Analysis: No. At the optimal solution that constraint has a slack of 158.727 logs. 


e Question: We have an informal commitment to buy at least 325 logs per month from 
supplier 1. Would including that constraint change the optimal solution? 
Analysis: No. Such a constraint would have the form 


Wer isie + We,1,1/8 + Wei1/16 + We i138 = 325 
Substituting the current optimal solution yields 
Wearsne + WEeas/3 + WeaneWE1 1/3 = 41.273 +0 +300 +0 
= 341.273 
> *825 


The constraint would not be active if it were included. 
e Question: Would a policy limiting green veneer purchase to $10,000 per month change 
the current optimal plan? 
Analysis: Yes. A new constraint enforcing this policy would have the form 


1.00x1/16.4 + 0.30%1/16,8 + 0.10X116,¢ + 2.20184 + 0.60%1/8,5 + 0.20x1/2,¢ < 10, 000 
Substituting the current plan yields 
1.00X3/16,4 + 9.30Xj)16,3 + 010K )16,¢ + 2.20x1/¢ 4 + 0.60x7) 5 + 0.20x1/¢ ¢ 
= 1.00(0) + 0.30(0) + 0.10(40,000) + 2.20(0) + 0.60(0) + 0.20(50,000) 
= 14,000 
# 10,000 


Our current optimal solution violates the proposed constraint. 


» ether each of the following modifications in the model will change this optimal 
solution. 


=) Dropping constraint 6x, — x2 + 2x3 > 20 
>) Dropping constraint 4x; — 3x3 < 15 
©) Adding constraint x; + x2 +.x3 <2 


4) Adding constraint 2x; + 7x2 +x3 <7 
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Analysis: 
(a) Substituting the optimal solution gives us 
6(3) — (0) + 2(1) = 20 


so that the constraint is active. Under principle | 7.39 |, dropping the constraint ma) 
change the optimal solution. 


(b) Substituting the optimal solution, we have 
46) — 30) = 9 = 15 
and the constraint is inactive. Dropping the constraint will not change the optima’ 
solution (principle | 7.39} ). 
(c) Substituting the optimal solution, we have 
(3)+ (0)+ (1) =4#2 
Since the constraint is violated, adding it will change the optimal solution (principle 
, 
(d) Substituting the optimal solution, we have 
2(3)+70)+(0)=7<7 


Adding the constraint will not change the optimal solution because it is alread 
satisfied (principle ) 


Dropping and Adding Variable What If’s 


The final category of “what if” questions that we can answer using Figures 7.4 ane 
7.5 involves dropping and adding variables. The dropping case is trivial: 

7.41 | An LP variable can be dropped without = oS solu lution 

only fits = = B00 ee 


For cramle, Table 7.5 implies that the optimal plan would remain ei ese if we 
eliminated 5-inch BC producing activity z1/2,2,c. Its optimal value zj. 5 ¢ = 0. 

Adding is a bit more complex. We need to decide whether the optimal value 
for an added variable would be zero, implying that it is just as well not to include = 
But this depends on whether current optimal dual prices v* produce a reduced objec- 
tive function coefficient with the proper sign (i.e., whether the main dual constrai= 
associated with the new variable is satisfied). 


Some examples (assuming the rest of the model is held constant): 


e Anew ;-inch AA plywood may be introduced that could be pressed in 0.25 hour usi=s 
two +-inch sheets of A veneer and one ;-inch sheet of B veneer. At what selling phx 
would this product enter the optimal plan? 
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Analysis: The corresponding dual constraint would be 


0.25v347 — 2v24 — 27 > 
Substituting the current dual solution yields 
0.25v}, — 2v3, — v3, = 0.25(51.418) — 2(—7.046) — (—10.925) 
~ $37.87 
At any price above this value, the new product would be profitable. 


e Question: A new supplier offers + ;-inch B green veneer at $0.40 per square foot. Would 
adding the possibility of purchasing his veneer change the optimal solution? 
Analysis: No. The only nonzero coefficient on this new activity would appear in 
the balance constraint for iz-inch B veneer. Thus the constraint would be 


Vig = — 0.40 


The present vj, = —0.201 satisfies this constraint, so it would be optimal to use none of 
the new activity. 


Suppose that a minimize eLP over nonnegative VanAbIES ite aie esa SIUHOn 
» = 0, x2 = 4, x3 = 2 and corresponding dual optimal solution v; = 1, v. = 0, 
; = 6. Determine whether each of the following modifications in the model will 

change this optimal solution. 


a) Dropping variable x, 
5) Dropping variable x2 


c) Adding a variable with coefficients 2, —1, and 3 in the main constraints and cost 


<) Adding a variable with coefficients 1, 0, and 2 in the main constraints and cost 50 
Analysis: 


2) In accord with principle , dropping x, will not change the optimal solution 
»ecause its optimal value = 0. 


>) Since x2 has a nonzero optimal value, dropping it would change the optimal 
lution (principle | 7.41} ). 
2) The main dual constraint for the new primal variable would be 
2v1 — lv2 + 3v3 < 18 
Substituting the optimal dual solution, we obtain 
2(1) — 1(0) + 3(6) = 20 £18 


Adding this variable would change the primal optimum because the current dual 


lution violates the new constraint (principle 


4) The main dual constraint for this new primal variable would be 
lv; +23 < 50 
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Substituting the optimal dual solution, we obtain 
1(1) + 2(6) = 13 < 50 


Since the constraint is satisfied, adding the new variable would not change the primal 


optimum (principle Ny 


7/7 BIGGER MODEL CHANGES, REOPTIMIZATION, 
AND PARAMETRIC PROGRAMMING 


Although standard LP sensitivity outputs lend insight on a host of minor “what if” 
questions, optimization by-products can go only so far. In this section we review 
briefly some of the practical limits and introduce more informative approaches re- 
quiring reoptimization. 


Ambiguity at Limits of the RHS and Objective Coefficient Ranges 


In Section 7.6 (principles and | 7.38|) we explained that the optimal primal 
solution provides reliable quantitative sensitivity information only as long as an ob- 
jective coefficient change stays within the range provided in LP outputs, and the 
optimal dual solution yields precise rates of change only if a right-hand-side change 
is restricted to the RHS range displayed. Beyond ie razes, we have only upper or 


Figure 7.6 illustrates how the ambiguity about rates of change, aay we first 
encountered in Section 7.2, occurs at the limits of RHS ranges. Three different 
versions of our venerable Two Crude refining model (7.4) are solved graphically using 
varying gasoline requirement RHS values b, = 2.0, 2.625, and 3.25, respectively. 

Computer output for the 2.0 case shows optimal dual slope vj = 20 holding 
within the RHS range 1.125 < b, < 2.625. A similar output for the b, = 3.25 choice 
indicates that a rate of change vj = 66.667 applies over the range 2.625 < b; < 5.100. 

One or the other of these dual and range outputs would print out if we solved 
with 6; exactly 2.625, but which one it would be is unpredictable. Either would 
mislead. The correct rate for an increase from b; = 2.625 is 66.667, and for a 
decrease it is 20, but neither possible output provides both. 


i ity ranges rates of 
iguous, with one value applying below the limit 
outputs ay show either value. : 


- optimal value change are 
and another above. Compu 


SAMPLE EXERCISE 7 29: INTERPRETING SENSITIVITY AT RANGE Limits 


Return to the Swedish Steel sensitivity plot of Figure 7.3(a). Determine from the 
plot the RHS ranges and optimal dual values that might result if an LP optimization 
code were invoked with the given RHS (a) 75.0; (b) 60.4. 

Analysis: 


(a) RHS value 75.0 falls within an unambiguous range. Output would show dual 
variable —3.38 and range [60.4, 83.3]. 


7.7 Bigger Model Changes, Reoptimization, and Parametric Programming 


BS, 
94 
3 
2 Optimal Lower Upper 
3 Dual Range Range 
20.000 1.125 2.625 
x 
| NC AGERES UES ae 
MO StTAEIS5 86 7H 181.9 
(a) b, =2.0 
2 
a. & 
5 - 
3 
= either 
P Optimal Lower Upper 
3 Dual Range Range 
4 20.000 Lot2'5) 20625 
a as] 
3 3 or 
? 2 
4 Optimal Lower Upper 
: Dual Range Range 
7 Ge 
MGs sees Saale eal 66.667 2.625 5.100 
(b) b, = 2.625 
*2 
at 
Optimal Lower Upper 
Dual Range Range 
66.667 2.625 5.100 


(c) b, = 3.25 


FIGURE 7.6 Sensitivity Ranges for Varying Two Crude Gasoline Demand 
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(b) RHS value 60.4 forms the boundary between two ranges. Output might show 
dual variable —3.38 and range [60.4, 83.3], or it might produce dual —4.98 with range 
[0.0, 60.4]. 


Connection between Rate Changes and Degeneracy 


A closer look at the three cases in Figure 7.6 shows what causes rates to change at 
b; = 2.625. Below that value, the optimal solution is defined by active gasoline and 
jet fuel requirement constraints 


0.3x; + 0.4x > by 
0.4x, + 0.2x2 > 1.5 


The corresponding dual slope is vj = 20. Above 2.625, the Venezuelan availability 
limit replaces jet fuel to give active set 


0.3x1 + 0.4x. > by 


x2 <6 


Rate vj = 66.667. 


At exactly b; = 2.625 all three constraints are active, but any two define primal 
solution x} = 0.75,x} = 6. This is the degenerate case of more constraints being 
active than are needed to settle the primal solution that we introduced in Section 
5.6. Rates of change displayed in sensitivity outputs will reflect whichever pair the 
optimization search happens to focus on. 

Many, perhaps most, large linear programs will have degenerate optimal solu- 
tions. It follows that we will often find that the given value of a model parameter 
also forms one or the other end of its range. Even if the base-case value lies strictly 
within a range, that interval of reliable quantitative sensitivity information is likely 
to be rather narrow. 


7,45_| Degeneracy, which is extremely common in large-scale LP models, lim- 
its the usefulness oF sensitivity bective cost from nal ae because 


Reoptimization to Make Sensitivity Exact 


The obvious remedy for weaknesses in sensitivity analysis based on optimization by- 
products is reoptimization—repeating the optimization search with changed model 
constants. For example, if we want to know how the optimal plan at Two Crude 
would change if gasoline demand increases from 2.0 to 3.5, we could just run again 
with the revised RHS. 


7.7 Bigger Model Changes, Reoptimization, and Parametric Programming 


practical avenue to g00 sensitivity analysis. 


Practical OR analyses rely heavily on running a variety of cases, and many lin- 
ear programming optimization codes provide for inputting several alternative RHSs 
or objective functions at the same time. Still, the approach has its limits. All combi- 
nations of even 10 parameter variations leads to 2'° = 1024 cases to try; 11 variations 
imply twice as many. 


Parametric Variation of One Coefficient 


To do sensitivity analysis by trying different cases, we must, of course, know what 
cases to try. For example, we may know we are interested in how sensitive optimal 
results are to changes in an RHS or objective coefficient, yet be unclear about exactly 
what values to consider. 

Parametric studies track the optimal value as a function of model inputs. Figures 
7.3 and 7.4 displayed just such parametric functions in our qualitative discussion of 
Section 7.2. Figure 7.7 adds another—the Two Crude optimal value as a function of 
the demand for gasoline. 


optimal 
value slope + 
(infeasible) ” 


250 


> RHS 


FIGURE 7.7 Parametric Variation of Optimal 
Value with Two Crude Gasoline Demand 


Many LP optimization codes include automatic features to produce such para- 
metric analysis, but we are now in a position to see how it can be done with the tools 
already at hand. Just four carefully chosen runs can construct the entire Figure 7.7 
curve. In particular, we employ the following cases and sensitivity outputs: 


Case RHS Dual Lower Range Upper Range 
Base model — 2.000 20.000 IDS: 2.625 
Variant 1 2.625+€ 66.667 2.626 5.100 
Variant 2 5.100 +¢€ +00 5.100 +00 
Variant 3 1.125-e 0.000 —0o 25 


where ¢€ is a small positive number. 
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Parametric analysis begins with the base model case of by = 2.000 and its 
optimal value of 92.5. The first line of table values shows that the slope or rate of 
change in optimal value at that point is vj = 20.000. Range information tells us that 
this slope holds from b; = 1.125 to 2.625. The result is the segment of the Figure 7.7 
curve passing through b; = 2.000, optimal value = 92.5. 

New rates of change arise only outside that range. Our first variation changes 
b, to 2.625 + €, just beyond the upper limit. The result is new slope 66.667 and an 
indication that it holds through upper limit b; = 5.100. Repeating for b; = 5.100 +e 
shows that the model is infeasible beyond 5.100. We complete the analysis by moving 
b, below the base-case range limit of 1.125. Reoptimization with bj = 1.125 —« 
indicates that the slope vj = 0.000 will hold for all further decreases. 


7Al : Parametric studies of = al value as a function of a single model RHS 


_ range. 


SAMPLE EXERCISE 7.30: PARAMETRICALLY ANALYZING ONE COEFFICIENT 
Return to the parametric plots of Figures 7.3 and 7.4. 


(a) Tabulate the separate optimizations that would have to be run to produce RHS 
parametric analysis Figure 7.3(a), beginning from the base the RHS value 75.0. 


(b) Tabulate the separate optimizations that would have to be run to produce objec- 
tive coefficient parametric analysis Figure 7.4(a), beginning from base cost 9.0. 


Analysis: 


(a) Proceeding from the figure according to principle produces the following 
cases: 


Case RHS Dual Lower Range Upper Range 
Base model 75.0 3.38 60.4 83.3 
Variant 1 83.3 +¢ 0.00 83.3 +00 
Variant 2 604+6 4.98 0.0 60.4 
Variant 3 0.0-€ —00 —0O 0.0 


(b) Proceeding from the figure according to principle produces the following 
cases: 


Case Cost Primal Lower Range Upper Range 
Base model 9.0 156.1 8.4 10.1 
Variant 1 10.1 +¢€ 109.56 10.1 19.9 
Variant 2 19.9 +6 82.03 19.9 29.2 
Variant 3 29.2 +€ 72.59 29.2 31.8 
Variant 4 31.8+6 72.21 31.8 Boe) 
Variant 5 35.5 +¢€ 68.43 35:5 +00 
Variant 6 8.4-¢€ 178.76 8.0 8.4 
Variant 7 8.0-e 400.98 —0O 8.0 
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Assessing Effects of Multiple Parameter Changes 


The greatest limitation of sensitivity studies we have explained so far is that all vary 
only one model input at a time. They increase or decrease a single RHS or objective 
coefficient, or add or drop a single variable or constraint. The implicit assumption is 
that all other data are held constant. 

Computations behind sensitivity printouts like those of Figures 7.4 and 7.5 
depend explicitly on this “single change” assumption. 


Unfortunately, one-at-a-time analysis often does not suffice. Many “what if” 
questions that arise in operations research studies involve multiple changes, with 
several model constants varying simultaneously. For example, we may wish to know 
the impact on Two Crude refinery’s optimal plan if demand for gasoline increases by 
some percent at the same time as demand for jet fuel increases at twice that percent 
because of an anticipated upturn in the economy; two RHSs would be changing at 
the same time. 

As with single variations, we can deal with a modest number of such multiple- 
change questions simply by reoptimizing with new coefficients. For example, a frac- 
tion @ increase in both gasoline and jet fuel demand at Two Crude would be imple- 
mented by a new run with 

= (1+ 4)2.0 
(7.14) 
by = (1+ 26)15 
and all other data as before. 


Parametric Multiple-RHS Change 


To see how to track the parametric effect of such multiple changes over a variety of 
change magnitudes 6, we need to think of right-hand sides in the form 


BEY — BP +6 AD; 


where each Ab; shows how much its RHS 5; is changing per unit variation, and @ 
defines the size of the step. In the example of (7.14), change components for the five 
model RHSs are 


Aby =2.0, Abz =2(1.5)=3, Ab3 = Aby= Abs =0 


Thinking of 6 as a decision variable, adding (6 Ab;) on the right-hand side of 
each constraint translates as —( Ab;)@ on the left. Therein lies the key insight. 


coefficients — Ab; hae pele the rates of change i in RHSs ae oS a ete axed 
by a new equality constraint. _. . 
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For example, the revised Two Crude model with varying rate of gasoline and jet fuel 


demand growth would be 


min +20x, +15x2 
s.t. +0.3x, +0.4x%. —20 >2 
+0.4x, +0.2x. —30 >15 
+0.2x, +0.3x2 > 0.5 
+ 1x4 59 
+ 1x. <6 
+10 =D¢ 


X1,X2 > 0,9 URS 


We can now proceed exactly as we did to construct Figure 7.7. New equality 


constraint 

0 = be 
reduces the task of parametric analysis to one of seeing how optimal value changes 
with RHS beg. 


Using RHS range outputs for that constraint leads us to the following sequence 
of runs and the parametric curve of Figure 7.8. 


Case RHS Dual Lower Range Upper Range 
Base model 2.000 20.000 1.125 2.625 
Variant 1 2.625+¢€ 66.667 2.626 5.100 
Variant 2 5.100 +€ +00 5.100 +00 
Variant 3 1.125 -e 0.000 —00 1.125 


Consider the linear program 


max +5x,; +2x» 

st. tly, +l» <3 
+ 1x4 ap 
X1,X2 > 0 


Show how to modify the model to parametrically analyze the effect of simultaneously 
increasing the first RHS and decreasing the second by the same amount. 


Analysis: The indicated change involves Ab; = +1, Aby = —1. Thus, applying 
principle , we may parametrically analyze the change by varying new RHS b: 
in the modified model: 


max +5x, +2x2 
st. +1x, +1x% -10 <3 
+ 1x4 +160 <2 
+16 =b; 


X1,x%2 = 0,8 URS 
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optimal 
value <—slope +0 
(infeasible) 
slope 
250 - 225.00 
slope 
145.00 ~~ 1 | 
150 Last 
| | 
| 
| 
I | 
50 4 2 ae >6 FIGURE 7.8 Parametric Multiple-RHS 
3 eee Change of the Two Crude Example 


Parametric Change of Multiple Objective Function Coefficients 


Tosee how to do the same sort of parametric multiple-change analysis when objective 
function coefficients are changing, we must think back to the relation between primal 
and dual. Parametric revision 


aa) : 
Ge =o +0 AG 
modifies the objective coefficients of the primal, which are the right-hand sides of 


the dual. Thus the analog of adding a new constraint —Ab column in the RHS case 
is the addition of a new constraint row —Ac to vary the c;. To be more specific, 


Again we illustrate using the Two Crude model (7.4). If we wish to explore 
parametrically the impact of a uniform crude price rise by a fraction 6, 


cre = 20+ 0Acy = 20 + 0(20) 
cy = 15+ 6Acp = 15 + (15) 
The revised primal model to manipulate is 
min +20x, +15x. +46x3 


st. +0.3x, +0.4x2 Si2) 
+0.4x, +0.2x2 > 1.5 
+0.2x, +0.3x2 > 0.5 

+ 1x1 <9 

+ 1x2 <6 


—20x, —-15x%. 4+1x3 =0 
Ny 0, Bess URS 
Negatives of rates Ac; form coefficients for a new equality constraint with RHS 
= 0.0, and new variable x3 appears in that constraint and the objective function. 
We can now determine parametric effects of our multiple changes by analyzing 
he effect on optimal value of changes in the single objective coefficient 9. The 
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process works because the main dual constraint corresponding to new variable x3 is 
V6 = 6 


Thus @ affects all other main dual constraints by a term — Acjvg = — Acj@ on the left 
hand side, which is equivalent to increasing at the same rate in the dual right-hanc 
side ¢. With primal and dual guaranteed to yield equal optimal values, manipulating 
the dual in this way does to the primal exactly what we want. 

Using objective function range outputs for parameter @ leads us to the following 
sequence of runs and the parametric curve of Figure 7.9. 


Case Coefficient 6 Optimal x; Lower Range Upper Range 
Base model 2.000 20.000 ils) 2.625 
Variant 1 2.625 +€ 66.667 2.626 5.100 
Variant 2 5.100 +€ +00 (infeasible) 5.100 +00 
Variant 3 1.125 -e 0.000 —00 125 
optimal 
value 

A 
250 slope 
6.67 


150 - 


1 
: 
FIGURE 7.9 Parametric Multiple 


50+ { *g gg §6Objective Coefficient Change of the 


3 = Two Crude Example 


Show how to modify the model in Sample Exercise 7.31 to parametrically analyze 
the effect of simultaneously increasing the first objective coefficient and decreasing 
the second at twice the rate. 


Analysis: The indicated change involves Ac} = +1, Acp = —2. Thus, applying 
principle , we may parametrically analyze the change by varying new objective 
coefficient 4 in the modified model: 


max +5x; +2x. +0%3 
s.t. tix, +15 <3 
fs 1x4, < 


—1xy, +2x. +13 =0 
Gin Moe 0, x3 URS 


EXERCISES 


7-1 As a result of a recent decision to stop produc- 
tion of toy guns that look too real, the Super Slayer 
Toy Company is planning to focus its production 
on two futuristic models: beta zappers and freeze 
phasers. Beta zappers produce $2.50 in profit for the 
company, and freeze phasers, $1.60. The company 
is contracted to sell 10 thousand beta zappers and 
15 thousand freeze phasers in the next month, but 
ali that are produced can be sold. Production of ei- 
ther model involves three crucial steps: extrusion, 
trimming, and assembly. Beta zappers use 5 hours 
of extrusion time per thousand units, 1 hour of trim- 
ming time, and 12 hours of assembly. Corresponding 
values per thousand units of freeze phasers are 9, 2, 
and 15. There are 320 hours of extrusion time, 300 
hours of trimming time, and 480 hours of assembly 
ume available over the next month. (An optimiza- 
ton output appears in Table 7.6.) 


2) Briefly explain how this problem can be mod- 
eled by the linear program: 


max 2500x; + 1600x2 
s.t. x, > 10 
xX, >15 
5x1 + 9x2 < 320 
1x, + 2x2 < 300 
12x, + 15x. < 480 
X1,%2. >0 


» Identify the resource associated with the ob- 
jective function and each main constraint in part 


(a). 
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(c) EQ] Identify the activity associated with each de- 
cision variable in part (a). 

(d) Interpret the left-hand-side coefficients of 
each decision variable in part (a) as inputs and 
outputs of resources per unit activity. 


7-2 Eli Orchid can manufacture its newest pharma- 
cutical product in any of three processes. One costs 
$14,000 per batch, requires 3 tons of one major in- 
gredient and 1 ton of the other, and yields 2 tons of 
output product. The second process costs $30,000 
per batch, requires 2 and 7 tons of the ingredients, 
respectively, and yields 5 tons of product. The third 
process costs $11,000 per batch, requires 9 and 2 tons 
of the ingredients, respectively, and yields 1 ton of 
product. Orchid wants to find the least costly way 
to produce at least 50 tons of the new product given 
that there are 75 tons of ingredient 1 and 60 tons 
of ingredient 2 on hand. (An optimization output 
appears in Table 7.7.) 


(a) Briefly explain how this problem can be mod- 
eled by the LP 
min 14x, + 30x, + 11x; 
st. 2x; + 5x2 + 1x3 > 50 
3x1 + 2x2 + 9x3 < 75 
1x, + 7x2 + 2x3 < 60 
X1,%2,%3 > 0 


(b) through (d) as in Exercise 7-1. 


TABLE 7.6 Optimization Output for Super Slayer Exercise 7-1 


lution value (max) = 77125.000 
*SiABLE SENSITIVITY ANALYSIS: 
== Optimal Bas Lower Upper Object Reduced Lower Upper 
Value Sts Bound Bound Coef Object Range Range 
21.250 BAS 0.000 +infin 2500.00 0.000 1280.000 PNET 
15.000 BAS D000: +infin 1600.00 0.000 “AnNbiihe St 2~ 000 
STRAINT SENSITIVITY ANALYSIS: 
eae Typ Optimal RHS Slack Lower Upper 
Dual Coef Range Range 
G 0.000 10.000 Pie 2o0 ei gnigaly a: 21.250 
Gy 1525-000 15.000 -0.000 0.000 24.000 
L -0.000 320.000 Tonal DO me eels 250) pokes’ 
L =0,000 300.000 248.750 ey 510) t+infin 
L 208.333 480.000 0.000 345.000 669.000 
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TABLE 7.7 Optimization Output for Eli Orchid Exercise 7-2 


(Gineraip ee slit Scala 
VARIABLE SENSITIVITY ANALYSIS: 


Solution value 


Name Optimal Bas Lower Upper Object Reduced Lower Upper 
Value Sts Bound Bound Coef Object Range Range 
Sila Som! 5.556 BAS 0.000 +infin 14.000 0.000 12.000 +infin 
x2 Lf HS BAS 0.000 +infin 30.000 0.000 -infin 35000 
x3 0.000 NBL O200 0S Santini Iles 010 0: 5.667 57935 t+infin 
CONSTRAINT SENSITIVITY ANALYSIS: 
Name Typ Optimal RHS Slack Lower Upper 
Dual Coef Range Range 
“el G7 5368 "50: 000) Sobou0 W2'a57 70.269 
c2 if ORO00= Sor O00 a2 71S sae 22. +infin 
(Oe) L == 60000 0.000 25.000 70.000 


7-3 Professor Proofis trying to arrange for the imple- 
mentation in a computer program of his latest oper- 
ations research algorithm. He can contract with any 
mix of three sources for help: unlimited hours from 
undergraduates at $4 per hour, up to 500 hours of 
graduate students at $10 per hour, or unlimited hours 
of professional programmers at $25 per hour. The 
full project would take a professional at least 1000 
hours, but grad students are only 0.3 as productive, 
and undergraduates, 0.2. Proof has only 164 hours 
of his own time to devote to the effort, and he knows 
from experience that undergraduate programmmers 
require more supervision than graduates, and grad- 
uates more than professionals. In particular, he esti- 
mates that he will have to invest 0.2 hour of his own 
time per hour of undergraduate programming,0.15 
hour of his time per hour of graduate programming, 
and 0.05 hour of his time per hour of professional 
programming. (An optimization output appears in 
Table 7.8.) 


(a) Briefly explain how this problem can be moc 
eled by the LP 


min 4x, + 10x2 + 25x3 

st. 0.2x; + 0.3x2 +.x3 > 1000 
0.2x,; + 0.15x+ 0.15x3 < 164 
X2 < 500 


X1,X2,x%3 > 0 


(b) [Q] through (d) [6] as in Exercise 7-1. 


7-4 The NCAA is making plans for distributins 
tickets to the upcoming regional basketball cham 
pionships. The up to 10,000 available seats will ™ 
divided between the media, the competing univer 
sities, and the general public. Media people am 
admitted free, but the NCAA receives $45 per ticks 
from universities and $100 per ticket from the ges 
eral public. At least 500 tickets must be reserved i= 
the media, and at least half as many tickets should = 


TABLE 7.8 Optimization Output for Professor Proof Exercise 7-3 


Solution value (min) = 24917.647 
VARIABLE SENSITIVITY ANALYSIS: 
Name Optimal Bas Lower Upper Object Reduced Lower Upper 
Value Sts Bound Bound Coef Object Range Range 
“xl «82.353. BAS 0.000. «Ss +infin 4.000 = 0.000. -4.444 5.000 
x2 0.000 NBL 0.000 Fini 10.000 22255 PACS t+infin 
x3 983.529 BAS 0.000 +infin 25.000 0.000 20.000 Silos 
CONSTRAINT SENSITIVITY ANALYSIS: 
Name Typ Optimal RHS Slack Lower Upper 
Dual Coef Range Range 
cl G 25.882 1000.000 -0.000 164.000 1093.333 
ee L 25.8382 164.000 0.000 .-150.,.000 1000.00 
es L 0.000 500.000 500.000 0.000 +infin 


*9 the competing universities as to the general pub- 

sc. Within these restrictions, the NCAA wishes to 

Sad the allocation that raises the most money. An 
ptimization output appears in Table 7.9. 
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(f) Minimize problem, 5w; —2w» < 11, increase -2. 

(g) [Q] Minimize problem, 4w, — 3w, > 15, de- 
crease -3. 

(h) Maximize problem, 3w; +4w2 < 17, decrease 3. 


TABLE7.9 Optimization Output for NCAA Ticket Exercise 7-4 


Solution value (max) = 775833.333 
ARIABLE SENSITIVITY ANALYSIS: 


ame Optimal Bas Lower Upper 
Value Sts Bound Bound 

7 500.000 BAS 0.000 +4infin 
2 3166.667 BAS 0.000 +infin 
3 CBBenes si. BAS 0.000 tinfin 

STRAINT SENSITIVITY ANALYSIS: 
me Typ Optimal RHS Slack 
Dual Coef 

mee) Le) -eisea7 0700000000 0.000 
2 G =3i6: {6/67 0.000 -0.000 
G -81.667 500.000 -0.000 


2) Briefly explain how this problem can be mod- 
eled by the LP 


max 45x. + 100x3 

s.t. X, +x +3 < 10, 000 
X2—5x%3 20 
xy > 500 


X1,%2,%3 >0 


>) through (d) as in Exercise 7-1. 


-5 For each of the following constraint coeffi- 
Sent changes, determine whether the change would 

shten or relax the feasible set, whether any implied 
“sange in the optimal value would be an increase or 
» decrease, and whether the rate of any such opti- 
mal value effect would become more or less steep if 
* varied with the magnitude of coefficient change. 
Assume that the model is a linear program, that all 

sriables are nonnegative, and that the constraint is 
»ot the only one. 


=) | Maximize problem, 3w; + w2 > 9, increase 
9. 

> Minimize problem, 5w, — 2w. < 11, decrease 
‘Hk 

=) &] Minimize problem, 4w; — 3w, > 15, de- 
crease 15. 

+ Maximize problem, 3w, + 4w2 < 17, increase 
AGE 

=) &] Maximize problem, 3w; +1» > 9, increase 
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Object Reduced Lower Upper 
Coef Object Range Range 
0.000 0.000 ~ -infin 81.667 
45.000 0.000 -200.000 100.000 
100.000 0.000 45.000 t+infin 
Lower Upper 
Range Range 
“500.000 +infin 
-4750.000 9500.000 
0.000 10000.000 


7-6 Determine whether adding each of the following 
constraints to a mathematical program would tighten 
or relax the feasible set and whether any implied 
change in the optimal value would be an increase or 
a decrease. Assume that the constraint is not the 
only one. 


(a) EQ] Maximize problem, 2w, + 4w> > 10. 
(b) Maximize problem, 14w, — w2 < 20. 

(c) [Q] Minimize problem, 45w; + 34w. < 77. 
(d) Minimize problem, 32w, + 67w2 > 49. 


7-7 For each of the following objective coefficient 
changes, determine whether any implied change in 
the optimal value would be an increase or a decrease 
and whether the rate of any such optimal value effect 
would become more or less steep if it varied with the 
magnitude of coefficient change. Assume that the 
model is a linear program and that all variables are 
nonnegative. 


(a) [] max 13w, + 4», increase 13. 
(b) max 5w; — 10w2, decrease -10. 


(c) &] min —Sw; + 17», increase -5. 
(d) min 29w; + 14w», decrease 29. 


7-8 Return to Super Slayer Exercise 7-1. 


(a) [IQ] Assign dual variables to cach main con- 
straint of the formulation in part (a), and define 
their meanings and units of measurement. 
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(b) [| Show and justify the appropriate variable- 
type restrictions on all dual variables. 

(c) [| Formulate and interpret the main dual con- 
straint corresponding to each primal variable. 

(d) [| Formulate and interpret an appropriate dual 
objective function. 

(e) Use optimal solutions in Table 7.6 to verify that 
optimal primal and dual objective function val- 
ues are equal. 

(f) [&] Formulate and interpret all primal comple- 
mentary slackness conditions for the model. 

(g) [| Formulate and interpret all dual comple- 
mentary slackness conditions for the model. 

(h) Verify that optimal primal and dual solutions in 
Table 7.6 satisfy the complementary slackness 
conditions of parts (f) and (g). 


7-9 Do Exercise 7-8 for the problem of Exercise 7-2 
using Table 7.7. 


7-10 [S| Do Exercise 7-8 for the problem of Exercise 
7-3 using Table 7.8. 


7-11 Do Exercise 7-8 for the problem of Exercise 
7-4 using Table 7.9. 


7-12 State the dual of each of the following LPs. 


(a) &} min 17x; + 29x. + x4 
st. 2x, + 3x2 + 2x3 + 3x4 < 40 
4x; + 4x. + x4 > 10 
3x3 —xX4 = 0 
Mires Kane. 


(b) max 44x; — 3x2 + 15x3 + 56x4 
s.t. xX $x. +43 +%4 = 20 
X1—X2 <0 
9x, — 3x2 +.x3—X4 > 25 
Nh teas eee) 
(c) B] max 30x, — 2x3 + 10x4 


s.t. 2x, — 3x2 + 9x4 < 10 
4x. — x3 > 19 
Xy +X. +%3=5 
x, >0,x3 <0 
(d) min 5x, +x. — 4x; 
st. xX, $X2+x3+xX4 = 19 
4x2 + 8x4 < 55 


X1 + 6x2 — x3 > 7 


X2,x3 > 0,%4 <0 
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(ec) B] max 2x; + 9x2 
s.t. 3w + 2x; — x2 > 10 
w—y<0 
x1 +3x%2.+y=11 
X1,X > 0 


19y, + 4y2 — 822 
s.t. yj +y2+21= 15 
Z1+5z2 <0 


(f) max 


Vis yn +22 24 
y1,¥2 20 
(g) 1 min 32x. + 50x3 — 19x5 
st. (15 =53) +x5 = 40 
12x; — 90x. + 14x4 > 18 
Kis Ut 
GAO ef SH, ego 
(h) max 10(x3 + x4) 
s.t. pe x; = 80 
Xj — 2xj41 > 0j=1,...,3 


X1,%2 = 0 


7-13 State (primal and dual) complementary slack- 
ness conditions for each LP in Exercise 7-12. 


7-14 Each of the following LP has a finite optime 
solution. State the corresponding dual, solve bom 
primal and dual graphically, and verify that optime 
objective function values are equal. 


(a) &l] max 14x; + 7x. 
s.L. 2x, + 5x2 < 14 
5x1 + 2x2 < 14 
X1,X%. => 0 
(b) min 4x; + 10x. 
st. 2x, +%. > 6 
x, < 10 
X1,X2 > 0 
(c) | min 8x,+11xy 


st. 2x, + 9x2 > 24 
3x1 xy ital 


X1,X%2 => 0 


7x1 + 9x2 
s.t. 4x, + 2x2. <7 
3x, + 7x2 < 19 


X1,X2 > 0 


d) max 


7-15 Compute the dual solution corresponding to 


zach of the following basic sets in the standard-form 
Tp 


max 6x1 - lx “+ 21x3 a 54x4 == 8x5 
s.t. 2x, + 5x3 + 7x5 = 70 
+3x2 + 3x3 — 9x4 + 1x5 =1 


iby. yeae eA) 


2) I fx, x9} 
B) {X1, x3} 
[O] {x2, x3} 


B) {X4, xs} 


7-16 Each of the followingis a linear program with no 
>timal solution. State the corresponding dual, solve 
~oth primal and dual graphically, and verify that 
»>enever primal or dual is unbounded, the other 
afeasible. 
fo] max 4x, +x 
SC. 2x; +x, > 4 
3x2 < 12 
X1,X2 = 0 
max 4x; + 8x2 
s.t. 4x, >8 
XH +% <1 
%1,x2 20 
Xd] min 10x; + 3x, 


s.t. Xb ho = 2 


—%,>5 
X1,X2 = 0 
min xX, — 2x2 
Sill, ease tia wee, SC) 
xX, — 5x. <5 
X1,X2 > 0 
O] min —3x,4+ 4x 
s.t. —xX, +2x >2 


X,—2Xx, >5 


egipecretl) 
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(f) max —20x, +15x2 
Sil. 3x, — 2x2 <6 
— 3x, + 2x, < -12 
XX: = 0 


7-17 For each of the following LPs and solution vec- 
tors, demonstrate that the given solution is feasible, 
and compute the bound it provides on the optimal 
objective function value of the corresponding dual. 


(a) &] min 30x, +2x. andx = (2,5) 
st. 4x, +x < 15 
5x1 —X%.>2 
15x; — 4x. = 10 
X1,%. > 0 
(b) max 10x, — 6x2 andx = (0,5) 
s.t. 12x, + 4x. > 8 
3x, —X) <5 
2x, + 8x2 = 40 


X1,X2 > 0 


7-18 Demonstrate for each linear program in Exer- 
cise 7-17 that the dual of its dual is the primal. 


7-19 Return to Exercise 7-1. Answer each of the fol- 
lowing as well as possible from the results in Table 
7.6. 


(a) [J Is the optimal solution sensitive to the ex- 
act value of the trimming hours available? At 
what number of hours capacity would it become 
relevant? 

(b) [| How much should Super Slayer be willing 
to pay for an additional hour of extrusion time? 
For an additional hour of assembly time? 

(c) K] What would be the profit effect of increasing 
assembly capacity to 580 hours? To 680 hours? 

(d) [|] What would be the profit effect of increas- 
ing the profit margin on beta zappers by $1500 
per thousand? What would be the effect of a 
decrease in that amount? 

(ec) [| Suppose that the model of Exercise 7-1 ig- 
nores packaging capacity because it is hard to 
estimate, even though each thousand beta zap- 
pers requires 2 hours of packaging, and each 
thousand freeze phasers requires 3 hours. At 
what capacity would packaging affect the cur- 
rent optimal solution? 
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[9] Suppose that Super Slayer also has a ninja 
nailer model it could manufacture that requires 
2 hours of extrusion, 4 hours of trimming, and 3 
hours of assembly per thousand units? At what 
profit per thousand would it be economic to pro- 
duce? 


Return to Eli Orchid Exercise 7-2. Answer 


each of the following as well as possible from the 
results in Table 7.7. 


(a) 
(b) 


(c) 


(d) 


(e) 


(f) 


(g) 


(h) 


7-21 


What is the marginal cost of production (per ton 
of output)? 

How much would it cost to produce 70 tons of 
the new pharmaceutical product? To produce 
100 tons? 

How much should Orchid be willing to pay to 
obtain 20 more tons of ingredient 1? How about 
ingredient 2? 

How cheap would the third process have to be- 
come before it might be used in an optimal so- 
lution? 

How much would the cost of the 50 tons of prod- 
uct increase if process 2 actually cost $32,000 per 
batch? If it cost $39,000? 

How much would the cost of the 50 tons of prod- 
uct decrease if process 1 cost $13,000 per batch? 
If it cost $10,000 per batch? 

Suppose that the engineering department is 
thinking about a new process that produces 6 
tons of product using 3 tons of each of the two 
original ingredients. At what cost would this 
new process be economic to use? 

Suppose that the three processes actually use 
0.1, 0.3, and 0.2 ton per batch of a third ingre- 
dient but we do not know exactly how much of 
itis available. Determine the minimum amount 
needed if the optimal primal solution in Table 
7.7 is not to change. 


Return to Exercise 7-3. Answer each of the fol- 


lowing as well as possible from the results in Table 


7.8. 
(a) 


(b) 


(c) 


[6] What is the marginal cost per professional- 
equivalent hour of programming associated 
with the optimal solution in Table 7.8? 

IS] How much would cost increase if 1050 
professional-equivalent hours of programming 
are required? How about 1100? 

[9] Does Professor Proof’s availability limit the 
optimal solution? How much would cost change 
if Professor Proof could devote only 150 hours 
to supervision? How about 100 hours? 


(d) 


() 


(f) 


(g) 


(h) 


I] How much would the hourly rate of gradu 
ate student programmers have to be reduced be- 
fore Professor Proof might optimally hire some 
| How much would project cost increase 
professional programmers cost $30 per hour 
If they cost $45? 

[§] Suppose that Professor Proof decides * 
require at least half of the total programme 
hours to go to students. Could this requireme= 
change the optimal solution? 

[€] Suppose that Professor Proof decides to = 
low unlimited hours of graduate student pr- 
gramming. Could this revision change the op 
mal solution? 

[IO] One of Professor Proof’s colleagues has = 
pressed interest in doing some of the program 
ming to earn outside income. At what price p= 
hour should Proof be interested if he estimate 
that the colleague would be 80% as efficient = 
a professional and require 0.10 hour of super 
sion per hour of work? 


7-22 Return to NCAA ticket Exercise 7-4. Answ= 
each of the following as well as possible from th: 
results in Table 7.9. 


(a) 
(b) 


(c) 


(d) 


(e) 


What is the marginal cost to the NCAA of eac® 
seat guaranteed the media? 

Suppose that there is an alternative arrang< 
ment of the dome where the games will ™ 
played that can provide 15,000 seats. How must 
additional revenue would be gained from t& 
expanded seating? How much would it be i= 
20,000 seats? 

Since television revenue provides most of th: 
income for NCAA events, another proposs 
would reduce the price of general public tics 
ets to $50. How much revenue would be Io 
from this change? What if the price were $3 
Media-hating coach Sobby Day wants 
NCAA to restrict media seats to 20% of thos: 
allocated for universities. Could this polio 
change the optimal solution? How abou 
10%? 

To accommodate high demand from studes 
supporters of participating universities, t< 
NCAA is considering marketing a new “scruns 
seat” that consumes only 80% of a regu= 
bleacher seat but counts fully against the “un 
versity > half public” rule. Could an optim= 
solution allocate any such seats at a ticket po& 
of $35? At a price of $25? 


7-23 Paper can be made from new wood pulp, from 
recycled office paper, or from recycled newsprint. 
New pulp costs $100 per ton, recycled office paper, 
$50 per ton, and recycled newsprint, $20 per ton. 
One available process uses 3 tons of pulp to make 1 
ton of paper; a second uses 1 ton of pulp and 4 tons of 
recycled office paper; a third uses 1 ton of pulp and 
12 tons of recycled newsprint; a fourth uses 8 tons of 
recycled office paper. At the moment only 80 tons 
of pulp is available. We wish to produce 100 tons of 
new paper at minimum total cost. 


a) Explain why this problem can be modeled as the 


LP 
min 100x; + 50x. + 20x; 


St. x4, =3yi + y2 + ys 


X2 = 4y2 + 8y4 

x3 = 12y3 

x, < 80 

4 

>= yj = 100 

j=l 

Rin yhse Vices eee) 


>) State the dual of the given primal LP. 

2) ©Enter and solve the given LP with the class 
optimization software. 

4) SlUse your computer output to determine a cor- 
responding optimal dual solution. 

2) OVerify that your computer dual solution is fea- 
sible in the stated dual and that it has the same 
optimal solution value as the primal. 

©) Use your computer output to determine the 
marginal cost of paper production at optimality. 

2) Use your computer output to determine how 
much we should be willing to pay to obtain an 
additional ton of pulp. 

©) Use your computer output to determine or 

bound as well as possible how much optimal cost 

would change if the price of pulp increased to 
$150 per ton. 

©) Use your computer output to determine or 

bound as well as possible how much optimal cost 

would change if the price of recycled office pa- 
per decreased to $20 per ton. 

©) Use your computer output to determine or 

bound as well as possible how much optimal cost 

would change if the price of recycled office pa- 
per increased to $75 per ton. 

© Use your computer output to determine or 

bound as well as possible how much optimal cost 
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would change if the number of tons of new pa- 
per needed decreased to 60. 

(1) @ Use your computer output to determine or 
bound as well as possible how much optimal cost 
would change if the number of tons of new pa- 
per needed increased to 200. 

(m) Use your computer output to determine how 

cheap recycled newsprint would have to become 

before the primal solution could change. 

(n) © An experimental new process will use 6 tons 
of newsprint and an undetermined number a 
tons of office paper. Use your computer output 
to determine how low « would have to be for 
the new process to be competitive with existing 
ones. 

(0) Use your computer output to determine 
whether a limit of 400 tons on recycled office pa- 
per would change the primal optimal solution. 

(p) & Use your computer output to determine 
whether a limit of 400 tons on recycled 
newsprint would change the primal optimal so- 
lution. 


7-24 Silva and Sons Ltd. (SSL)! is the largest coconut 
processor in Sri Lanka. SSL buys coconuts at 300 
rupees per thousand to produce two grades (fancy 
and granule) of desiccated (dehydrated) coconut for 
candy manufacture, coconut shell flour used as a 
plastics filler, and charcoal. Nuts are first sorted into 
those good enough for desiccated coconut (90%) 
versus those good only for their shells. Those dedi- 
cated to desiccated coconut production go to hatch- 
eting/pairing to remove the meat and then through 
a drying process. Their shells pass on for use in flour 
and charcoal. The 10% of nuts not suitable for desic- 
cated coconut go directly to flour and charcoal. SSL 
has the capability to hatchet 300,000 nuts per month 
and dry 450 tons of desiccated coconut per month. 
Every 1000 nuts suitable for processing in this way 
yields 0.16 ton of desiccated coconut, 18% of which 
is fancy grade and the rest granulated. Shell flour is 
ground from coconut shells; 1000 shells yield 0.22 ton 
of flour. Charcoal also comes from shells; 1000 shells 
yield 0.50 ton of charcoal. SSL can sell fancy desic- 
cated coconut at 3500 rupees per ton over hatchet- 
ing and drying cost, but the market is limited to 40 
tons per month. A contract requires SSL delivery of 
at least 30 tons of granulated-quality desiccated co- 
conut per month at 1350 rupees per ton over hatch- 
eting and drying, but any larger amounts can be sold 


‘R. A. Cabraal (1981), “Production Planning in a Sri Lanka Coconut Mill Using Parametric Linear 


*ogramming,” Interfaces, 11:3, 16-23. 
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at that price. The market for shell flour is limited 
to 50 tons per month at 450 rupees each. Unlimited 
amounts of charcoal can be sold at 250 rupees per 
ton. 


(a) Explain how this coconut production planning 
problem can be modeled as the LP 
max 3500s; + 1350s. + 450s3 
+ 25054 — 300p1 = 300p2 


st.  0.10p; — 0.90p2 =0 
0.828, — 0.1859 =0 
Pr < 300 
S1 +58) < 450 
Sy < 40 
$2 > 30 
$3 < 50 
0.16p; — 51 — 82 —X() 


0.11p; + 0.11p2 — 0.50s3 — 0.2254 = 0 


P1, P2, 81, 82; 83, 54 =0 


(b) State the dual of your primal linear program. 
(c) Enter and solve the given primal LP with the 
class optimization software. 

(d) Use your computer output to determine a cor- 

responding optimal dual solution. 

(c) Verify that your computer dual solution is fea- 

sible in the stated dual and that it has the same 

optimal solution value as the primal. 

(f) On the basis of your computer output, deter- 
mine how much SSL should be willing to pay 
to increase hatcheting capacity by 1 unit (1000 
nuts per month). 

(g) On the basis of your computer output, deter- 
mine how much SSL should be willing to pay 
to increase drying capacity by 1 unit (1 ton per 
week). 

(h) On the basis of your computer output, deter- 
mine or bound as well as possible the profit im- 
pact of a decrease in hatcheting capacity (thou- 
sands of nuts per month) to 250. Do the same 
for a capacity of 200. 

(i) On the basis of your computer output, deter- 
mine or bound as well as possible the profit im- 
pact of an increase in hatcheting capacity (thou- 
sands of nuts per month) to 1000. Do the same 
for a capacity of 2000. 
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(j) &The company now has excess drying capacity. 
On the basis of your computer output, deter- 
mine how low it could go before the optimal 
plan was affected. 

(k) & The optimum now makes no shell flour. On 
the basis of your computer output, determine at 
what selling price per ton it would begin to be 
economical to make and sell flour. 

(1) & On the basis of your computer output, de- 
termine or bound as well as possible the profit 
impact of a decrease in the selling price of gran- 
ulated desiccated coconut to 800 rupees per ton. 
Do the same for a decrease to 600 rupees. 

(m) © On the basis of your computer output, de- 
termine or bound as well as possible the profit 
impact of an increase to 400 rupees per ton in the 
price of charcoal. Do the same for an increase 
to 600 rupees. 

(n) On the basis of your computer output, deter- 
mine whether the primal optimal solution would 
change if we dropped the constraint on drying 
capacity. 

(0) & On the basis of your computer output, deter- 
mine whether the primal optimal solution would 
change if we added a new limitation that the to- 
tal number of nuts available per month cannot 
exceed 400,000. Do the same for a total not to 
exceed 200,000. 


7-25 Tube Steel Incorporated (TSI) is optimizing 
production at its 4 hot mills. TSI makes 8 types 
of tubular products which are either solid or hol- 
low and come in 4 diameters. The following two 
tables show production costs (in dollars) per tube of 
each product at each mill and the extrusion times (in 
minutes) for each allowed combination. Missing val- 
ues indicate product—mill combinations that are not 
feasible. 


Unit Cost 
Product Mill1 Mill2 Mill3 Mill4 

0.5 in. solid 0.10 0.10 — 0.15 
1 in. solid 0.15 0.18 — 0.20 
2 in. solid 0.25 0.15 — 0.30 

4 in. solid 0.55 0.50 — — 
0.5 in. hollow — 0.20 0.13 0.25 
1 in. hollow - 0.30 0.18 0.35 
2 in. hollow — 0.50 0.28 0.55 
4 in. hollow — 1.0 0.60 = 


Bazaraa, Mokhtar S., John J. Jarvis, and Hanif Sher- 
ali (1990), Linear Programming and Network Flows, 
Wiley, New York. 

Chvatal, VaSek (1983), Linear Programming, W. H. 
Freeman, San Francisco. 
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Unit Time (e) Verify that your computer dual solution is fea- 
Product Mill1 Mill2 Mill3  Mill4 sible in the stated dual and that it has the same 
istiniealil 0.50 0.50 - 0.10 optimal solution value as the primal. 
1 in. solid 0.60 0.60 a: 0.30 (f) Use your computer results to determine the 
2 in. solid 0.80 0.60 _— 0.60 marginal cost of producing each of the eight 
4 in. solid 0.10 1.0 — —_— products. 
Se uenow tz = ee oe (g) Use your computer results to explain why the 
han feller pa 16 080 0.80 policy of operating all mills at least one shift is 
4 in. hollow _ 2.0 1.0 = costing the company money. 
(h) Two options being considered would open 
mills 3 or 4 on weekends (i.e., add up to 16 extra 
Yearly minimum requirements for the solid sizes (in hours to each of 3 shifts over 50 weeks). Tak- 
thousands) are 250, 150, 150, and 80, respectively. ing each option separately, determine or bound 
For the hollow sizes they are 190, 190, 160, and 150. as well as possible from your computer results 
The mills can operate up to three 40-hour shifts per the impact these changes would have on total 
week, 50 weeks a year. Present policy is that each production cost. 
mill must operate at least one shift. (i) Another option being considered is to hire 
(a) F ea Gina mate ba d young industrial engineers to find ways of re- 
Beit: ee ec oa E nae a ducing the unit costs of production at high-cost 
out S fe Ae the Book Bon rone er mill 4. For each of the 6 products there taken 
using the decision variables separately, use your computer results to deter- 
Xp,m thousands of units of product p produced mine to what level unit costs would have to be 
annually at mill mm reduced before there could be any change in the 
: : : optimal production plan. 
pain constraints shale Rowe tye emai min: (j) SA final pair of options being considered is to 
imum time constraints, followed by a system of F ; 2 f 
; : 3 install equipment to produce 4-inch solid and 
4 maximum time constraints, followed by a sys- : ; : 
Pr of 8 demand conbtialnte 4-inch hollow tubes at mill 4. The new equip- 
Ey Gtate the dual of your prea LP model ment would produce either product in 1 minute 
c) & Enter and solve your primal linear program et = ave ese Seana TS aaa 
ME acs oes Beet mine the unit production cost that would have 
P v6 to be achieved to make it economical to use the 
(d) Use your computer output to determine a cor- ane 
; : ; new facilities. 
responding optimal dual solution. 
SUGGESTED READING 


Luenberger, David G. (1984), Linear and Nonlinear 
Programming, Addison-Wesley, Reading, Mass. 
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Multiobjective 
Optimization and Goal 
Programmin 


Most of the methods of this book address optimization models with a single objec- 
tive function—a lone criterion to be maximized or minimized. Although practical 
oroblems almost always involve more than one measure of solution merit, many can 
>= modeled quite satisfactorily with a single cost or profit objective. Other criteria 
are either represented as constraints or weighted in a composite objective function 
*o produce a model tractable enough for productive analysis. 

Other applications—especially those in the public sector—must simply be 
treated as multiobjective. When goals cannot be reduced to a common scale of cost or 
cenefit, trade-offs have be addressed. Only a model with multiple objective functions 

s satisfactory, even though analysis will almost certainly become more challenging. 

This chapter introduces the key notions and approaches available when such 
multiobjective analysis is required. Emphasis is on efficient solutions, which are 
»ptimal in a certain multiobjective sense, and goal programming, which is the most 
commonly employed technique for dealing with multiobjective settings. 


8.1 MULTIOBJECTIVE OPTIMIZATION MODELS 


As usual, we begin our investigation of multiobjective optimization with some exam- 
oles. In this section we formulate three cases illustrating the broad range of applica- 
Sons requiring multiobjective analysis. All are based on real contexts documented 
= published reports. 


EXAMPLE 8.1: BANK THREE INVESTMENT 


Every investor must tradeoff return versus risk in deciding how to allocate his or 
ser available funds. The opportunities that promise the greatest profits are almost 
= ways the ones that present the most serious risks. 

Commercial banks must be especially careful in balancing return and risk be- 
cause legal and ethical obligations demand that they avoidundue hazards, yet their 
=oal as a business enterprise is to maximize profit. This dilemma leads naturally to 
multiobjective optimization of investment that includes both profit and risk criteria. 
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Our investment example! adopts this multiobjective approach to a fictitious 
Bank Three. Bank Three has a modest $20 million capital, with $150 million in de- 
mand deposits (checking accounts) and $80 million in time deposits (savings accounts 
and certificates of deposit). 
Table 8.1 displays the categories among which the bank must divide its capital 
and deposited funds. Rates of return are also provided for each category together 
with other information related to risk. 


TABLE 8.1 Bank Three Investment Opportunities 


Return Liquid Required Risk 


Investment Rate Part Capital Asset? 

Category, j (%) (%) (%) 
1: Cash 0.0 100.0 0.0 No 
2: Short term 4.0 99.5 0.5 No 
3: Government: 1 to 5 years 4.5 96.0 4.0 No 
4: Government: 5 to 10 years 25 90.0 5.0 No 
5: Government: over 10 years 7.0 85.0 75S) No 
6: Installment loans 10.5 0.0 10.0 Yes 
7: Mortgage loans 8.5 0.0 10.0 Yes 
8: Commercial loans 92 0.0 10.0 Yes 


We model Bank Three’s investment decisions with a decision variable for each 
category of investment in Table 8.1: 


x; 4 amount invested in category j ($ million) j=1,...,8 


Bank Three Example Objectives 


The first goal of any private business is to maximize profit. Using rates of return 
from Table 8.1, this produces objective function 


max 0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg 


It is less clear how to quantify investment risk. We employ two common ratio mea- 
sures. 

One is the capital-adequacy ratio, expressed as the ratio of required capital for 
bank solvency to actual capital. A low value indicates minimum risk. The “required 
capital” rates of Table 8.1 approximate U.S. government formulas used to compute 
this ratio, and Bank Three’s present capital is $20 million. Thus we will express a 
second objective as 


min 3, (0.005x2 + 0.040.x3 + 0.050x4 + 0.075xs (capital-adequacy) 
+ 0.100x6 + 0.100x7 + 0.100xg) 


Another measure of risk focuses on illiquid risk assets. A low risk asset/capita! 
ratio indicates a financially secure institution. For our example, this third measure 
of success is expressed as 


min 7 (%+x7+xs) —_ (risk-asset) 


1Based on J. L. Eatman and C. W. Sealey, Jr. (1979), “A Multiobjective Linear Programming Mode 
for Commercial Bank Balance Sheet Management,” Journal of Bank Research, 9, 227-236. 
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Bank Three Example Model 


Tocomplete a model of Bank Three’s investment plans, we must describe the relevant 
constraints. Our example will assume five types: 


1. Investments must sum to the available capital and deposit funds. 
2. Cash reserves must be at least 14% of demand deposits plus 4% of time deposits. 


3. The portion of investments considered liquid (see Table 8.1) should be at least 47% of 
demand deposits plus 36% of time deposits. 


4. Atleast 5% of funds should be invested in each of the eight categories, for diversity. 


5. At least 30% of funds should be invested in commercial loans, to maintain the bank’s 
community status. 


Combining the 3 objective functions above with these 5 systems of constraints 
completes a multiobjective linear programming model of Bank Three’s investment 


problem: 
max 0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg 
min =: (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x6 + 0.100x7 + 0.100xg) 
min 35 (X6 +X7 + Xe) (risk-asset) 
St. Xp +++) + xg = (20 + 150 + 80) (invest all) (8.1) 
x, = 0.14(150) + 0.04(80) (cash reserve) 
1.00x; + 0.995x2 + 0.960x3 + 0.900x,4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
a; > 0.0520 + 150+ 80). forallj =1,..4,8 (diversification) 
xg > 0.30(20 + 150 + 80) (commercial) 


OGh, cal ig S00 


EXAMPLE 8.2: DYNAMOMETER RING DESIGN 


Multiobjective optimization also occurs in the engineering design of almost any prod- 
uct or service. Competing performance measures must be balanced to choose a best 
design. 

We will illustrate with the design of a simple mechanical part, an octagonal 
ring used in dynamometer instrumentation of machine tools. Figure 8.1 depicts the 
critical design variables: 


w 4 width of the ring (in centimeters) 
t A thickness of the outer surface (in centimeters) 
r 4 radius of the two openings (in centimeters) 


It also shows upper and lower limits allowed for the three dimensions. 


2Based on N. Singh and S. K. Agarwal (1983), “Optimum Design of an Extended Octagonal Ring 
>y Goal Programming,” International Journal of Production Research, 21, 891-898. 
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5.00 < w < 10.00 
0.10<¢t < 2.00 
1.25 < r < 20.00 


aeae — 


FIGURE 8.1 Dynamometer Ring Design Example 


Dynamometer Ring Design Model 


To model this machine part optimization, we must characterize performance in terms 
of decision variables w, t, and r. One consideration is sensitivity. We would like 
the instrument to be as sensitive as possible. Analysis of the relevant strains and 
deflections shows that sensitivity can be represented as 


air ni 
max Eve (sensitivity) 
where £ = 2.1 x 10°, Young’s modulus of elasticity for the ring material. 

Another measure of effectiveness is rigidity. Increased rigidity produces greater 
accuracy. Again considering critical strains and deflections, rigidity can be modeled 
as 

3 


Ewt b 36] 
max _ (rigidity) 


Combining these two objectives with upper and lower bounds on the variables 
yields a multiobjective optimization model of our ring design problem: 


0.7r 


max Ewtt (sensitivity) 
Ewt? ee 
max —3 (rigidity) (82) 
5.0 < w < 10.0 
0.1 <t<2.0 
1.25 <r < 20.0 


Notice that this model is a multiobjective nonlinear program. It can, however, be 
handled by linear programming if we change variables to In(w), In(f), and In(r) and 
maximize the logarithms of the two objective functions (See Section 14.9). 


EXAMPLE 8.3: HAZARDOUS WASTE DISPOSAL 


Many, perhaps most, government planning problems are multiobjective because they 
involve competing interests and objectives that are difficult to quantify in monetary 


8.1 Multiobjective Optimization Models atT: 


terms. We illustrate with the planning of disposal sites for dangerous nuclear 
wastes. 
Figure 8.2 depicts the source points and potential disposal sites we assume in 
_ our fictitious version of the problem. Waste material is generated at the seven sources 
as nuclear power plants and other reactors use up fuel. The goal of the study is to 
choose 2 of the 3 possible disposal sites to receive the waste. 


= waste source 


= possible disposal 
site 


FIGURE 8.2. Hazardous Waste Example Sources and Disposal Sites 


When the system is ready, wastes will be trucked from sources to disposal sites 
over the public highway system. Conflicting objectives arise in routing materials to 
the sites. Costs will be reduced, and in-transit time of wastes minimized if we ship 
along shortest routes from source to disposal site. But population densities must also 
be considered. Routes through densely populated areas threaten more people and 
incur greater risk of traffic mishaps. 

Table 8.2 shows parameters 


s; & amount of waste expected to be produced at source i 
dist * distance from source i to site j along route k 
Dijk 5 population along route k from source i to site j 


for 2 possible routes from each source i to each possible disposal site. We wish to 
minimize some combination of distance and population to select the best sites and 
routes. 


3Based on C. ReVelle, J. Cohon, and D. Shobrys (1991), “Simultaneous Siting and Routing in the 
Disposal of Hazardous Wastes,” Transportation Science, 25, 138-145. 
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TABLE 8.2 Hazardous Waste Example 


ae Site j=1 Sitej=2 _Sitej=3 Supply 
i ol kee k=l k=2,, hol pha? 
1 Distance 2002 280-850 1090 900 1100.12 


Population 50 TS) ~ 300 80 400 190 

2 Distance 400 530 © 730 860 450 600 0.5 
Population —_105 60 380 210 350 160 

3. Distance 600 F35m—550 600-210 240 0.3 
Population 300 130" 520 220) 270 140 

4 Distance 900 1060 450 570. ~—-:180 360 0.7 
Population 620 410 700 430 800 280 

5 Distance 600 640 = 390 440 360 510 0.6 
Population 205 180 440 370 680 330 

6 Distance 900 1240 100 120 640 800 0.1 
Population 390 125 80 30 ~=—-800 410 

7 Distance 1230 ©1410 =400 460 1305 1500 0.2 
Population 465 310 180 105 1245 790 


Hazardous Waste Disposal Model 


To model this shipping and site selection problem, we require the two sorts of decision 
variables typical of facility location models: 


ae i if site i is opened 
yi = : 
- 0 otherwise 
Migr = amount shipped from source i to site j along route k 


Then a multiobjective integer linear programming model is 


Sr) 
min y i» > Genie (distance) 
i=1 j=1 k=1 
RAGS 
min bw ye » Paik (population) 
i=1 j=l k=1 
5 *’ 
s.t. ye >i ces; SF (sources) (8.3) 
j=l k= 4 
3 
ae (2 sites) 
j=l 
Xi, ik S Sid; email die Oey: Spl ie es ee Peyaeadls: 2) 
Xe =O Ea eng oe fe rey) rk Le 
yj=Oorl rule en 


The first main constraints assure that all material arising at each source is shipped 
somewhere, and the second selects 2 sites. Switching constraints makes sure that a 
site is opened before any material can be shipped there. 


8.2 EFFICIENT POINTS AND THE EFFICIENT FRONTIER 


The familiar notion of an “optimal” solution becomes somewhat murky when an 
optimization model has more than one objective function. A solution that proves best 
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by one criterion may rate among the poorest on another. In this section we develop 
the concept of efficient points and the efficient frontier, also known as Pareto optima 
and nondominated points, which help to characterize “best” feasible solutions in 
multiobjective models. 


Efficient Points 


A feasible solution to an optimization model cannot be best if there are others that 
equal or improve on it by every measure. Efficient points are ones that cannot be 
dominated in this way. 


Efficient points are also termed Pareto optimal and nondominated. 
We can illustrate with the dynamometer ring design model of Section 8.1: 


max Be (sensitivity) 
1 x 105) (wi) ‘ 


Gwaee 
max ee = 2) (rigidity) (8.4) 


5.0 < w < 10.0 
OLS 2:0 
125s r= 20.0 


Consider solution (w, t, r) = (5, 0.1, 20). No feasible point has higher sensi- 
tivity, because the sole variable r in the numerator of the sensitivity objective is at 
its upper bound, and both w and ¢ in the denominator are at lower bounds. But 
there are feasible points with higher rigidity; we need only decrease radius r. Even 
though it can be improved in one objective, the point is efficient. Any such change 
to improve rigidity results in a strict decrease in sensitivity. 

Contrast with the solution (w®, t@,r®) = (7,1,3). Its values in the two 
objective functions are 


ie 0.7(3) - 
sensit = = 1.429 x 10 
sensitivity 1x 105 Dae x 
2.1 x 10®)(7)(1)3 
fipidity Ae eet 10° 


(3) 


This feasible point cannot be efficient because it is dominated by (w®, r9,r®) = 
(7, 0.9, 2.7). The latter has the same rigidity and superior sensitivity 1.587 x 10S", 


Identifying Efficient Points Graphically 

When a multiobjective optimization model has only two decision variables, we can 
see graphically whether points are efficient. Consider, for example, the simple mul- 
tiobjective linear program 
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max +3x, + 1x 

max —1x; + 2x2 

st. xy +x <4 (8.5) 
O<x, <3 
0<x% <3 


plotted in Figure 8.3. Feasible solution x = (2,2) of part (a) is efficient and the 
x = (3, 0) of part (b) is not. 


x2 


better points 


efficient x = (2,2) 


1 2 3 4 
(a) Efficient Point 


x2 


better points 


%1 FIGURE 8.3 Graphical Characterization of 
Efficient Points 


(b) Dominated Point 


How can we be sure? In each case we have plotted the contours of the two 
objective functions that pass through the point. Other solutions with equal or better 
values of both objective functions must belong to the dark shaded areas bounded 
by those contours. In part (a) no such point is feasible. In part (b), others, such as 
x’ = (3, 1), satisfy all constraints and thus dominate. 
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the point, v 
objectives. 


Determine whether each of the following points is efficient in model (8.5) and Fig- 
ure 8.3. 


a) x = (1,3) 
(b)x = (1, ) 
Analysis: We apply principle in the following plots: 


(a) (b) 


(a) In plot (a) the set of possibly dominant points, which is bounded by the contours, 
contains no other feasible point. Solution x = (1, 3) is efficient. 


‘b) In plot (b) the set of possibly dominant points, which is bounded by the contours, 
contains numerous feasible solutions. Solution x = (1, 1) cannot be efficient. 


Efficient Frontier 


When confronting a multiobjective optimization model, it seems natural to demand 
an efficient solution. But we have seen that very simple examples can have many 
efficient points. Some will score better by one criterion; others will evaluate superior 
by another. 

To deal with such conflicts, we would often like to generate and consider a range 
of efficient solutions. The entire set is called the efficient frontier. 


on model is the col- 
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Plots in Objective Value Space 


The term efficient frontier comes from an alternative way to plot solutions to multi- 
objective models. Instead of using axes corresponding to the decision variables, we 
plot in objective value space with axes for the different objective functions. 

Figure 8.4 illustrates for our dynamometer ring design example (8.4). Every 
feasible solution for the model corresponds to a point in this plot, with horizontal di- 
mension equal to its sensitivity, and vertical equal to its rigidity. For example, efficient 
point (w®, tr) = (5, 0.1, 20) produces point (1.333 x 1074, 1.312) at the lower 
right. Dominated point (w®, t®, r©) = (7, 1, 3) graphs as (1.429 x 10-7, 5.444 x 109). 


values for w = 10, f =2,r =1.25 
efficient frontier 


100,000 


10,000 
values for w=7,t=1,r =3 


values for w =5,t=0.1,r = 20 a oe 


Ty OT eet pa el ell leeN pete Tal ypaly st 


1e-08 1e-07 1e-06 1e-05 0.0001 
sensitivity 


FicurRE 8.4 Efficient Frontier for Dynamometer Ring Example 


The efficient frontier forms the boundary of the region defined by objective 
values for feasible points. Every efficient point lies along this boundary because no 
further progress is possible in one objective function without degrading another. On 
the other hand, dominated points plot in the interior; other feasible solutions along 
the boundary are equal in all objectives and superior in at least one. 


Constructing the Efficient Frontier 


When a multiobjective optimization model has only a few objectives, it is often prac- 
tical to construct efficient frontier curves such as Figure 8.4. We have only to para- 
metrically vary specified levels of all but one objective while optimizing the other. 


The process parallels the parametric sensitivity analysis of Section 7.7. 
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Table 8.3 details the computations used to construct the efficient frontier of the 
dynamometer ring example in Figure 8.4. We begin by maximizing the two objectives 
separately. Maximizing sensitivity without regard to rigidity produces the first point 
(1.333 x 1074, 1.312) in the table. Then maximizing rigidity without considering 
sensitivity yields to the last point (1.042 x 1078, 8.6 x 107). 


TABLE 8.3 Efficient Frontier of the Dynamometer 


Ring Example 

Sensitivity Rigidity 

Objective Objective Efficient Point 

Ww t r 

1.333 x 104 1.312 5.00 0.100 20.00 
6.776 x 10° 10! 5.00 0.100 10.16 
3.145 x 105 102 5.00 0.100 4.72 
1.460 x 105 105 5.00 0.100 2.19 
5.510 x 10-6 10* 5.00 0.123 1.25 
1.187 x 10° 10° 5.00 0.265 1.25 
2.557 x 107 10° 5.00 0.571 1.25 
5.510 x 10-8 107 5.00 1.230 1.25 


1.042 x 10°8 8.6 x 107 10.00 2.000 1.25 


We now know the range of relevant rigidity values. The remaining points in 
Table 8.3 are obtained by maximizing sensitivity subject to a constraint on rigidity. 
For example, the values for rigidity 10° result from solving 

0.7r 
(2.1 x 10°) (we?) 


6 i} 
CoE ee fetes) 


max (sensitivity) 


s.t. 


r3 
5.0 < w < 10.0 
0.1 <7 = 2:0 
1.25 <r < 20.0 


SAMPLE EXERCISE 8.2: 


Return to example model (8.5) of Figure 8.3 and construct an objective value space 
diagram of its efficient frontier like the one in Figure 8.4. 


Analysis: We begin by separately maximizing the first and second objectives to ob- 
tain solutions with objective values (10,—1) and (3,6), respectively. Now solving the 


LP 
max +3x;+ 1x 
s.t. —Ix, +2x. >6 
xX, +x. <4 
O<x, <3 


O0<x <3 
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for several values of @ € [—1, 6] produces the following plot: 


second objective 


first objective 


§,3 PREEMPTIVE OPTIMIZATION AND WEIGHTED 
SUMS OF OBJECTIVES 


In a typical multiobjective model of realistic size, especially one with more than two 
objectives, the range of efficient solutions can be enormous. Explicit construction of 
an efficient frontier like Figure 8.4 is computationally impractical. 

To obtain useful results, we must reduce the multiobjective model to a sequence 
of single objective optimizations. In this section we explore two of the most straight- 
forward ways: preemptive (or lexicographic) optimization and weighted sums. 


Preemptive Optimization 


Although a model may have several objective criteria, they are rarely of equal im- 
portance. Preemptive optimization takes them in priority order. 


Preemptive Optimization of the Bank Three Example 


We can illustrate with the Bank Three example, model (8.1). That model has 3 
objectives: profit, capital-adequacy ratio, and risk-asset ratio. 

Suppose we decide that the third, risk-asset, objective is the single most impor- 
tant one. Preemptive optimization would then begin by minimizing risk-asset ratio 
in the single objective linear program 


min > (x6 + x7 + x8) (risk-asset) 
St. Xy +--+ +xg = 204 150+ 80 (invest all) 
x1 => 0.14(150) + 0.04(80) (cash reserve) 


1.00x; + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 
+ 0.850x5 = 0.47(150) + 0.36(80) 


8.3 Preemptive Optimization and Weighted Sums of Objectives 


x; => 0.05(20 + 150+ 80) forallj=1,...,8 (diversification) 
xg > 0.30(20 + 150 + 80) (commercial) 
Xeon Ree 
An optimal solution allocates funds (in millions of dollars) 
eq OOS LT Sew one 125 
Bede Sree heel Spee a ee Kms LO.) 


with optimal risk-asset ratio 5.0, profit $11.9 million, and capital-adequacy ratio 0.606. 
Next we introduce a constraint keeping risk-asset ratio at the optimal 5.0 and 
maximize the second priority, profit objective: 


(8.6) 


max 0.040x2 + 0.045x3 + 0.055x4 + 0.070xs5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092x 
s.t. 4 (x6 + x7 +xs) < 5.0 (risk-asset) 
xy +--+ +xg = 20+ 150+ 80 (invest all) 
x1 > 0.14(150) + 0.04(80) (cash reserve) 
1.00x; + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
x; = 0:05(20 + 150+ 80) ‘forallj =1,...,8 (diversification) 
Xg > 0.30(20 + 150 + 80) (commercial) 
Ginvigm ap Reon. 
The result is (millions of dollars) 
4 = 242, x5 =125, 145125, af =125 
KE B8.3 i oe 12.5, as = 12.5, 2 a, = 75.0 
with optimal risk-asset ratio still 5.0, but profit now $17.2 million, and capital-adequacy 
ratio 0.890. 
Notice that the character of the optimum has changed significantly from solu- 
tion (8.6). Considerable funds have been shifted from x; = cash, to x5 = long-term 
2overnment bonds in order to increase profit. 


Finally, we come to the capital-adequacy objective function. Imposing a new 
constraint for profit, we solve 


min = (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x6 + 0.100x7 + 0.100x8) 


(8.7) 


s.t. = (x6 + X7 +X) < 5.0 (risk-asset) 
0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg > 17.2 
xy +++ +xg = 20+ 150 + 80 (invest all) 
x, > 0.14(150) + 0.04(80) (cash reserve) 
1.00x; + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
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x; = 0.05(20 + 150 + 80) 
xg = 0.30(20 + 150 + 80) 
Kifiy, cso, XR = O 
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forallj= Wess (diversification) 


(commercial) 


This time solution (8.7) remains optimal. The capital-adequacy ratio cannot be de- 
creased without worsening other objectives. 


SAMPLE EXERCISE 8.3: SOL 


Consider the multiobjective mathematical program 


max 
max 
s.t. 


Wi 
2w, + 3w2 
Ww, <3 


Wy tw. <5 
W1,W2 => 0 


Solve the model graphically by preemptive optimization, taking objectives in the 
sequence given. 


Analysis: Graphic solution produces the following plot: 


wo 


extra constraint added 
in second maximization 
> 

I 


preemptive 
optimum 


Wy 


A first optimization maximizes w, subject to the given constraints. Any of the 
solutions along the line segment from (3, 0) to (3, 2) is optimal. 
Next we impose an extra constraint 


Ww, >3 


and maximize the second objective. The result is final preemptive solution w = (3, 2 


Preemptive Optimization and Efficient Points 


One advantage of the preemptive approach to multiobjective optimization is the 
it results in solutions that cannot be improved in one objective without degradins 
another. 
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“he preemptive process requires that we try objective functions in turn, trying to 
‘prove one without worsening the others. When we finish, no further improvement 
= possible. As usual, infeasible and unbounded cases can produce complications, but 
“e typical outcome is an efficient point. 


Preemptive Optimization and Alternative Optima 


Although it usually will produce an efficient point, a moment’s reflection on the 
=recemptive optimization approach will reveal a major limitation: 


8.7 After one objective function has 
ing ofa multiobjective model, solution 
tive Optima in the first. _ 


tained i sibsequent stapes must all 


That is, preemptive optimization places very great emphasis on the first objective 
addressed, with all later steps limited to alternative optima in the highest-priority 
bjective. 
The Bank Three computations above showed considerable change from initial 
»ptima (8.6) to final (8.7) because the first, risk-asset objective 


min a (x6 + x7 + x8) 


simits many alternative optima among the unmentioned x;,...,x5. But with cases 
where alternative optima are rare, the preemptive approach becomes essentially one 
» optimizing a priority objective while ignoring all the others. 


Weighted Sums of Objectives 


An alternative scheme for dealing with multiobjective models that permits more 
salanced handling of the objectives is simply to combine them in a weighted sum. 


‘ens orient all objectives in the same direction, and weights reflect their relative 
™mportance. 


SAMPLE EXERCISE 8.4 


Form a single Weipntede -sum Sones epreelve from each of thie eollowitie col- 
=ctions of objective functions. Indicate whether weights y; should be positive or 
»egative and whether the composite objective should be maximized or minimized. 
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(a) min 42w, +3w2 —1w3 
max +4w,; —2w2 
max +1w. +13 


(b) min +3w, — 12 
min 44w, +2w2 +4 9w3 


Analysis: We apply principle | 8.8}. 
(a) Using weights 71, ..., 73, the weighted objective is 
max (271 + 4y2)w1 + (3y1 — 2y2 + ly3)w2 + (—1yi + lys)w3 
This maximize composite form requires that y, < 0, y > 0, and 73 > 0. 
(b) Using weights 7, and ~, the weighted objective is 
min (37 + 4y2)w1 + (—1y1 + 2y2)w2 + Or) 


This minimize composite form requires that y; > 0 and y2 > 0. 


Weighted-Sum Optimization of the Hazardous Waste Example 
Hazardous waste planning model (8.3): 


hee Sah 
eee = pas ye dij kXijnk (distance) 
i=1 j=l k=1 
i. ee 
ania pS ») >a PijkXi,j.k (population) 
i=1 j=l k=1 
cee re 
ae = be Xijk=5S; %t=l,...,7 (sources) 
j=l k=1 
3 
do =2 (2 sites) 
j=1 
Xijk S Sid; = ene, (a i — len sh ie = il, 2 
Mik 20 i=1,...,7; j=l,...,3; k=1,2 
yj =Vorl pens 


illustrates a setting where weighted-sum analysis of a multiobjective model can us: 
fully be applied. The first objective minimizes the shipping distance to chosen d= 
posal sites. The second minimizes the population exposed along the way. Eact 
source is provided 2 alternative routes to each potential disposal site, one denote. 
k = 1, emphasizing short distance, and the other k = 2, avoiding population. 

Since both objective minimize, we may produce a single composite objecti= 


by applying weights ;, y2 > 0 and minimizing the result (principle ): 
Tues 


2 
min a (vidijix + VoD i,j,k) Xi,j,k (composite) 
El Eee 


Table 8.4 illustrates the impact for some different combinations of weight 
With comparatively high weight 7; on distance, the optimization routes almost =! 


8.4 Goal Programming 


TABLE 8.4 Weighting Objectives in the Hazardous 
Waste Example 


_—— a a 
Total Total Optimal 


Weight Ton-Miles Ton-Miles _Ton-Population Sites 
Nn k=1 k=2 


10 1 1155 0 1155 1334.5 153 
10 5) 754-591 1345 782.5 Use) 
10 10 1046 404 1450 607.5 12 
SyuelO 440 1114 1554 533.0 IS) 
Al) OS AS 1715 468.5 ks} 


along short routes k = 1. As population is given greater relative weight, activity shifts 
to the longer routes that avoid population except when the optimal sites change at 
1 = ¥% = 10. Eventually, all shipping is along the safer routes. Confronted with a 
-ange of such alternatives, decision makers could decide an appropriate balance. 


Weighted-Sum Optimization and Efficient Points 


Although offering more flexibility in trading off objectives, using weighted totals still 
assures an efficient solution. 


Tosee why this is true, we need only consider the nature of a weighted-sum objective: 
weight objective 4 weight objective 
1 1 value 2 2 value Ey 


if weight objective 
P p value 
“vith signs as in construction [8.8], any solution that can improve in one objective 


without degrading the others would also score better in the weighted objective. Only 
sn efficient point could be optimal. 


8.4 GOAL PROGRAMMING 


“Multiobjective models of complex problems assume that we always want more of 
=verything—lower values of objectives being minimized at the same time as higher 
values of criteria being maximized. Notice that this is independent of how much we 
may already have achieved on one or another objective. For example, a multiob- 
=ctive model would keep the same priority or weight on a minimize cost objective 
=aether or not we already have in hand a solution with extraordinarily low cost. 

In this section we explore the goal programming alternative, which is con- 
“ructed in terms of target levels to be achieved rather than quantities to be max- 
maized or minimized. It is probably more realistic to assume that the importance 

‘any criterion diminishes once a target level has been achieved. We will certainly 
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see that it is easier to implement. That is why goal programming is by far the mos* 
popular approach to finding good solutions in multicriteria problem settings. 


Goal or Target Levels 


Goal program modeling of a multiobjective optimization begins by asking decisio= 
makers to specify new data: goal or target levels for each of the criteria used t 
evaluate solutions. 


Goal Form of Bank Three Example 


To illustrate, return to our Bank Three example of Section 8.1. 


max  0.040x2 + 0.045.x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x65 + 0.085x7 + 0.092x¢ 
min a (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x5 + 0.100x7 + 0.100x) 
min 35 (x6 +.X7 + Xs) (risk-asset) 
s.t. xX, +-++-+xg = (20+ 150 + 80) (invest all) 
x1 > 0.14(150) + 0.04(80) (cash reserve) 
1.00x, + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 
+ 0.850x5 > 0.47(150) + 0.36(80) 
x; > 0.05(20+ 150+ 80) forallj=1,...,8 (diversification) 
xg = 0.30(20 + 150 + 80) (commercial) 
Molise 5 08: 0 


Here solutions are evaluated on three criteria: profit, capital-adequacy ratic 
and risk-asset ratio. Assume that instead of seeking ever higher levels of the firs 
criterion and lower values of the last two, we set some goals: 

profit > 18.5 
capital-adequacy ratio <0.8 (88 
risk-asset ratio < 7.0 


Then the problem might be specified in goal format as 


goal 0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg > 18.5 
goal mn (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x¢ + 0.100x7 + 0.100xg) < 0.8 
goal 4 (x6 + x7 +x) > 7.0 (risk-asset) 
st. xy +++++xg = (20+ 150 + 80) (invest all) (8s 
x1 => 0.14(150) + 0.04(80) (cash reserve) y 
1.00x; + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
x; = 0.05220 + 150+ 80) forallj=1,...,8 (diversification) 
Xg > 0.30(20 + 150 + 80) (commercial) 
Ms in XQ =O 


8.4 Goal Programming 


Soft Constraints 
Goals in statement (8.9) may be thought of as soft constraints. 


cas 
ay still be violated in — 


The more familiar hard constraints still determine what solutions are feasible, leaving 
the soft ones to influence which solutions are preferred. 


Deficiency Variables 


Once target levels have been specified for soft constraints, we proceed to a more 
familiar mathematical programming formulation by adding constraints that enforce 
goal achievement. However, we cannot just impose the constraint that each objective 
meet its goal. There may be no solution that simultaneously achieves the desired 
levels of all soft constraints. Instead, we introduce new deficiency variables. 


In the 3-objective Bank Three example (8.9), we enforce goal levels (8.8) with 
deficiency variables 


d, = amount profit falls short of its goal 
dy = amount capital-adequacy ratio exceeds its goal 
ds = amount risk-assct ratio exceeds its goal 


Expressing Soft Constraints in Mathematical Programs 


Goal and other soft constraints can now be expressed in the usual (hard) mathemat- 
ical programming format with deficiency variables allowing violation. 
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For example, the three goals in Bank Three model (8.9) lead to the following 
main (and variable type) constraints: 
0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg + dy > 18.5 
= (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x6 + 0.100x7 + 0.100xg) — dy < 0.8 
55 (X6 +.X7 +X) — d3 < 7.0 (risk-asset) 
d,, dz, d3 = 0 
The first keeps profit at least 18.5 or makes up the difference with deficiency variable 
d,. The other two main constraints force the capital-adequacy and risk-asset ratios 
below our target value unless the corresponding deficiency variables are nonzero. 


Notice that nonnegativity constraints are required. We want the deficiency t 
compute as zero if the target is achieved. 


Return to the multiobjective mode 


max wy} 

max 2w,+3w2 

s.t. Ww, <3 
wit+w2 <5 
W1,W2 => 0 


of Sample Exercise 8.3, and assume that instead of seeking to maximize the two 
criteria, we decide to seek a target level of 2.0 on the first and 14.0 on the second 
Introduce deficiency variables and formulate new goal constraints to model thes= 
soft constraints as those of a linear program. 


Analysis: We apply construction using deficiency variables 


d, 4 undersatisfaction of the first goal 


dz A undersatisfaction of the second goal 


Then the new linear constraints are 


W1 +d, >2.0 
2w,+3w2. +d, > 14.0 
d,,d,=0 


Goal Program Objective Function: Minimizing (Weighted) 
Deficiency 


Having modeled undersatisfaction of goals with deficiency variables, we complete = 
formulation by minimizing violation. 
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Often, all deficiency is weighted equally. 


Goal Linear Program Model of the Bank Three Example 


Using equal goal weights in our Bank Three example (8.9) produces the goal linear 
program 


min dj +a)4+d43 (total deficiency) 
st. 0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 
+ 0.105x6 + 0.085x7 + 0.092xg + d, > 18.5 
a (0.005x2 + 0.040x3 + 0.050x4 + 0.075x5 (capital-adequacy) 
+ 0.100x6 + 0.100x7 + 0.100xg) — d < 0.8 
5h (X6 +.X7 +-xg) — ds < 7.0 (risk-asset) 
Xi +--+ +xg = 20+ 150+ 80 (invest all) (8.10) 
x1 > 0.14(150) + 0.04(80) (cash reserve) 
1,00x1 + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
x; = 0.057220 + 150+ 80) forallj =1,...,8 (diversification) 
Xg > 0.30(20 + 150 + 80) (commercial) 
Mi, as. Xe = 0 


d,, dz, d3 > 0 


The goals have been expressed with new constraints involving deficiency variables. 
All original constraints are retained. 


Alternative Deficiency Weights in the Objective 


Table 8.5 shows an optimal solution to equal-weighted goal LP in column (1). Notice 
now it seeks only the $18.5 million goal for profit and the 7.0 goal for risk-asset ratio. 
Jnce corresponding deficiency variables d; and d> are driven to = 0.0, effort can be 
Sirected toward the remaining capital-adequacy goal. 

There is no requirement that deficiency weights be equal. In our Bank Three 
=xample, the magnitude of the capital-adequacy ratio is much smaller than that of the 
sther two objectives. Thus better results might be obtained by a different weighting 
“at scales deficiencies more equally. 

Column (2) of Table 8.5 shows the effect of multiplying the capital adequacy 
seight by 10. Now the profit slips below its $18.5 million goal (to $17.53 million), 
vat the capital-adequacy ratio deficiency is reduced. 
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TABLE 8.5 Goal Programming Solution of the Bank Three Example 


qd) (2) (3) (4) (5) 
Equal Unequal Preempt Preempt Preempt 
Weights Weights Profit Profit, CA One Step 


Profit goal weight 1 1 il 0 10,000 
Cap-adequacy goal weight 1 10 0 1 100 
Risk-asset goal weight 1 il 0 0 1 
Extra constraints d,= = 
Profit 18.50 17.53 18.50 18.50 18.50 
Deficiency, d; 0.00 0.97 0.00 0.00 0.00 
Capital- adequacy ratio 0.928 0.815 0.943 0.919 0.919 
Deficiency, d3 0.128 0.015 0.143 0.119 0.119 
Risk-asset ratio 7.000 7.000 7.097 7.158 7.158 
DEBS, d3 0.000 0.000 0.097 0.158 0.158 
Cash, x} 24.20 24.20 24.20 24.20 24.20 
Short term, x3 16.03 48.30 12.50 19.73 19.73 
Government: 1-5, x3 12.50 12.50 12.50 12.50 12.50 
Government: 5-10,.x4 12.50 12.50 12.50 12.50 12.50 
Government: over 10, X5 44.77 12.50 46.37 37.91 37.91 
Installment, X6 52.50 52.50 41.08 55.67 55.67 
Mortgages, x7 12.50 12.50 12.50 12.50 12.50 
Commercial, xg 75.00 75.00 88.36 75.00 75.00 


SAMPLE EXERCISE 8.6: FORMULATING GOAL PROGRAMS 


Assigning equal weights to violations of the two goals, formulate the goal linear 
program corresponding to the constraints and target values of Sample Exercise 8.5 


Analysis: Including the goal constraints of Sample Exercise 8.5 and minimizing tota’ 
deficiency as in construction produces the goal linear program 


min d,+d) 

Sit: w, +d, > 2.0 
2w, + 3w2 + do > 14.0 
w,<3 
wi +w2 <5 
W1,W2 > 0 
di,d, >0 


Notice that all original constraints have been retained. 


Preemptive Goal Programming 


The preemptive optimization of Section 8.4 (definition [8.5]) takes objective func 
tions in order: optimizing one, then a second subject to the first achieving its optim= 
value, and so on. A similar preemptive goal programming approach is often adopte: 
after criteria have been modeled as goals. 


=S 15 | Preemptive or lexic = programming. considers ae one at 


8.4 Goal Programming 


Preemptive Goal Programming of the Bank Three Example 


Columns (3) and (4) of Table 8.5 illustrate this preemptive variant of goal program- 
ming. The first was obtained by focusing entirely on the profit goal. Using the 
objective function 


min d 1 


we concern ourselves only with achieving the desired profit of $18.5 million. Column 
3) shows a distribution of investments that achieves that goal, albeit at the cost of 
violating both the others. 

Now we turn to the capital-adequacy ratio goal. After adding the extra con- 
straint 


d, =0 


‘0 keep the profit goal fully satisfied, we address capital adequacy with objective 
function 


min d) 


The result [column (4) of Table 8.5] shifts investments to improve the capital-adequacy 
ratio while continuing to achieve a profit of $18.5 million. 
To complete the preemptive goal programming solution of this example, we 
should now address the last risk-asset ratio objective by 
min d3 


subject to extra constraints 


d, =0.0, d, =0.119 


The effect is to seek a solution coming closer to the risk-asset ratio goal without 
sing ground on either of the other two. For this example the resulting solution is 
“Se same as column (4). 


Seturn to the weighted goal program of Sample Exercise 8.6. 


*) Formulate the first model to be solved if we wish to give highest priority to fulfilling 
e first goal. 


>) Assuming that the first goal can be completely achieved in the model of part (a), 
~smulate a second model to seek satisfaction of the second goal subject to satisfying 
the first. 


Analysis: We execute optimization sequence | 8. 
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(a) The first model to be solved emphasizes deficiency in goal 1: 
min d, 
st. witd, > 2.0 
2w, + 3w2 + do = 14.0 
wy <3 
witw2 <5 
W1,W2 > 0 
d,,d,>0 


(b) Assuming that deficiency in the first goal can be completely eliminated, the seco 
optimization to be solved is 


min d) 
st. wit+d, > 2.0 
2w, + 3w24+ do = 14.0 
w, <3 
wi+w. <5 
Wi, W2 = 0 
dy, > 0 
d, = 0 
New constraint d,; = 0 requires that the first goal be fully satisfied. Within © 
limitation, we minimize violation of the second goal. 


Preemptive Goal Programming by Weighting the Objective 


If a given model has a number of objective functions (or goals), preempt 20 


programming in the successive optimization manner of principle |. 


rather tedious. Fortunately, the same effect can be achieved in a saple optimiza 
by choosing appropriate weights. 


We may implement the profit first, capital-adequacy ratio second, risk- 
ratio third prioritization of our Bank Three goal programming model (8.10) in 
way by using the objective function 


min 10,000d; + 100d, + 1d; (preemptive-weighted deficiency) 


Column (5) of Table 8.5 shows that this weighting achieves the same solution as 
one obtained sequentially in column (4). 


8.4 Goal Programming 


E GOAL 


»rcemptive priority sequence in the example of Sample Exercise 8.7. 


min 100d; + d 
st. w,+d, > 2.0 
2w, +3w2 + d> > 14.0 
w, <3 
wi t+w2<5 
W1,W2 > 0 


d;,d,>0 


Practical Advantage of Goal Programming in Multiobjective 
Problems 


Although the same techniques can be used for any soft constraints, we have seen 
from our treatment of the Bank Three example how goal programming can provide 
a practical tool for dealing with multiobjective problem settings. If decision makers 
can be persuaded to specify target levels for their various objectives—and often that 
is not easy—modeling as a goal LP or ILP or NLP reduces the multiobjective analysis 
to standard mathematical programming form. 


Goal Programming and Efficient Points 


We have seen in Section 8.2 that efficient points (definition [8.1]) provide the clos- 
est analog to an optimal solution in multiobjective settings because they cannot be 
improved in one objective without degrading another. Unfortunately, goal program- 
ming reformulation of a multiobjective model cannot always guarantee a solution 
with this desirable property. 


To see the difficulties that can occur, suppose that we modify Bank Three goal 
LP (8.10) by ignoring the capital-adequacy objective. The result is the two-goal form 
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min di+d, (total deficiency) 
st. 0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 (profit) 

+ 0.105x6 + 0.085x7 + 0.092xg + d; > 18.5 

+ 0.100x6 + 0.100x7 + 0.100xg) — dp < 0.8 


x (X6 + X7 +X) — d3 < 7.0 (risk-asset) 

x1 +---+xg = (204 150 + 80) (invest all) 

x1 > 0.14(150) + 0.04(80) (cash reserve) (8.11) 
1.00x; + 0.995x2 + 0.960x3 + 0.900x4 (liquidity) 


+ 0.850x5 > 0.47(150) + 0.36(80) 
x; = 0.057220 + 150+ 80) forallj =1,...,8 (diversification) 
xg = 0.30(20 + 150 + 80) (commercial) 
Mires: 008 0 
di, d3 >0 
Solving gives the goal programming optimum 
HPS 2420 eS diy tag = 15,3 YE Sas 
4, = 483, Xe — 4430, Xe = 125, ae = 83.15 
with profit $18.5 million and risk-asset ratio 7.00. 
Both goals are satisfied completely. Still, solution (8.12) is not efficient because 


either of the objectives can be improved without worsening the other. For example. 
the efficient solution (an alternative goal program optimum) 


xf=24.2, x$=125, x8=125, x§ = 12.5 
x2 = 51.33, x= 49.47, x*=12.5, x% = 75.0 


continues to produce $18.5 million in profit but reduces the risk-asset ratio to 6.849 

What makes such nonefficient goal programming solutions occur? A moment's 
reflection will reveal that the problem is goals that are fully satisfied. Both objectives 
achieved their target values in (8.12). Once such a solution drives the corresponding 
deficiency variable = 0, there is nothing in the goal programming formulation te 
encourage further improvement. Progress stops without reaching an efficient point 


(8.12 


(8.13 


min wy 

max w2 

st. O<w, <2 
O<w2 <1 


(a) Introduce deficiency variables to formulate a goal program with target values © 
for the first objective function and 2 for the second. 


8.4 Goal Programming 


(b) Show that (wi, w2) = (1, 1) is optimal in the goal program of part (a) but not 
efficient in the original multiobjective form. 


‘c) Show that (w1,w2) = (0, 1) is also optimal in the goal program of part (a) but 
that it is efficient in the original multiobjective form. 


Analysis: 


(a) Using deficiency variables d, in the first objective and d> in the second, the 
required goal program is 


min di +d, 

SG wy —d, <1 
W2 + dz >2 
O<w, <2 
O<w2<1 
d,,d, >0 


(b) At solution (w;, w2) = (1, 1), the first goal is satisfied (d; = 0) and the second 
misses by d; = 2—1=1. However, no feasible solution can make w> > 1, so the 
solution is optimal in the goal program. 

Even though this solution is optimal in the goal program, it is not efficient. We 
may feasibly make the first objective less that 1.0 while retaining 1.0 in the second. 


c) The solution (w1, w2) = (0, 1) is feasible and achieves the same minimum total 
deficiency as the solution of (b). But this alternative optimum is efficient. 


Modified Goal Program Formulation to Assure Efficient Points 


In many applications, model constraints and objectives are so complex that goal 
programming’ s potential for producing non-efficient points never arises. Fortunately, 
there is an easy correction when it is a concern. 


We may illustrate with a modified Bank Three goal program (8.11). Efficient 
solution (8.13) results when we change the goal program objective to 
min d; +43 
— 0.001 (0.040x2 + 0.045x3 + 0.055x4 + 0.070x5 + 0.105x¢ 


0. = 
+ 0.085x7 + 0.092xg) + (x6 + x7 + Xg) 


Multiple 0.001 of the maximize profit a rot has been subtracted and the same 
multiple of the risk-asset ratio added to the standard deficiency objective. 
For the same reason that weighted-sum approaches produce efficient solutions 


principle| 8.9), any result from this modified goal program can be optimal only if no 
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original objective can be improved without degrading another. Still, the underlying 
goal program resolution of the multiple objectives will not be affected as long as the 
weight on original objectives is kept small. 


Produce a revised version of the goal program in Sample Exercise 8.9(a) that must 
yield an efficient solution in the original multiobjective optimization. 


Analysis: Following construction , we introduce a small multiple of the original 


objective functions. Using multiplier 0.001, the result is 


min d, +d) +0.001(w1) — 0.001(w2) 
S.t. wi — dy <1 

W2 + dz > 2 

O<w, <2 

O<w2 <1 

d,, do >0 


The multiplier is positive for the first, minimize objective, and negative on the maxi- 
mize objective. The goal program objective of minimizing deficiency is still dominant, 
but new terms now assure that alternative optima will resolve to efficient points. 


EXERCISES 


8-1 The sketch that follows shows the District 88 
river system in the southwestern United States. 


(20% loss) hydroelectric 
con 
communities irrigation 


All water arises at mountain reservoir R1. The esti- 
mated flow is 294 million acre-feet. At least 24 mil- 
lion acre-feet of the water is contracted to go directly 
from R1 to nearby communities, but the communi- 
ties would accept as much as could be supplied. The 
remainder flows through the desert to a second reser- 
voir, R2, losing 20% to evaporation along the way. 
Some water at R2 can be allocated for irrigation of 
nearby farms. The remainder flows over a hydro- 
electric dam and passes downstream. To maintain 
the equipment, the flow over the dam must be at 
least 50 million acre-feet. District 88 sells water to 
communities at $0.50 per acre-foot and to irrigation 
farmers at $0.20 per acre foot. Water passing through 


the hydroelectric dam earns $0.80 per acre-foot. The 
district would like to maximize both water supplied 
for irrigation and sales income. 


(a) Briefly explain how this problem can be mod- 
eled by the multiobjective linear program 


max Xx) : 

max 0.50x; +0.20x2 +0.80x3 

Grains a +1.25x2 +1.25x3 = 294 
xy = 24, x2 = 0, x3 > 50 


(b) | & Use the class optimization software to 
sketch the efficient frontier for the model of part 
(a) in an objective value graph like Figure 8.4. 

(c) | & Use the class optimization software to 
show that the optimal solutions taken separately 
for each objective do not coincide. 


8-2 A semiconductor manufacturer has 3 different 
types of silicon wafers in stock to manufacture its 3 
varieties of computer chips. Some wafer types can- 
not be used for some chips, but there are two alter- 
natives for each chip. The table that follows shows 


a= cost and on-hand supply of each wafer type, the 
sumber of each chip needed, and a score (0 to 10) 

‘the appropriateness match of each wafer type for 
making each chip. 


Wafer Unit On 
Type Chip Match Cost Hand 
1 2 3 
1 dl 8 — 15 500 
2 10 — 6 25 630 
3) — 10 10 30 710 


Need 440 520 380 
eee 
The company would like to minimize the total cost 
of the wafers used while maximizing the total match 
score. 


(a) Briefly explain how this problem can be mod- 
eled by the multiobjective LP 


min 15x11 + 15x12 + 25x 4 
+ 25x23 + 30x32 + 3023.3 
max 7x1; + 8x, + 10x21 


+ 6x23 + 10x32 + 10x33 
Sut X11 $%2 S 500 
X21 +X2,3 < 630 
X32 +233 < 710 
X11 +%2,1 = 440 
X12 + x32 = 520 
X2,3 +33 = 380 
all x;; > 0 
(b) and (c) as in Exercise 8-1. 


8-3 A national commission is deciding which mili- 
tary bases to close in order to save at least $85 mil- 
lion per year. The table that follows shows projected 
annual savings (millions of dollars) for each of five 
possible closings, together with the implied percent 
loss of military readiness and the number of civilian 
workers (thousands) who would lose their jobs. 


Base 
1 2 3 4 5) 
Savings DAP 2 we tase, 184 80 


Readiness 10 04 14 18 2.0 
Workers Zo S446 4) 


Each base must be left open or completely closed, 
and the commission wants to meet the required 
savings while minimizing both readiness loss and 
unemployment. 


SEL, a cs MMe ee 


Exercises 401 
(a) Briefly explain how this base-closing problem 
can be modeled by the multiobjective ILP 

min 1.0x; + 0.4x2 + 1.4x3 
+ 1.8x4 + 2.0x5 
2.5x, + 5.4x2 + 4.6x3 
+ 4.2x4 + 14.4x;5 
St. 24x, + 29x, + 45x53 
+ 34x4 + 80x5 > 85 
OG = Olote hay alte 25 


(b) and (c)&] as in Exercise 8-1. 


min 


8-4 A garden superstore sells four kinds of fertilizer 
from 20 pallet spaces aligned along a fence. Weekly 
demands for the four fertilizers are 20, 14, 9, and 5 
pallets, respectively, and each pallet holds 10 bags. 
The sales counter is at the beginning of the fence, 
and every bag sold must be carried from its pallet to 
the counter. Whenever all pallets for a product are 
empty, the space allocated for that fertilizer is refilled 
completely using a fork truck from secondary stor- 
age. Store managers need to decide how many pallet 
spaces to assign each product to minimize both the 
number of times per week such refilling will be nec- 
essary and the total carrying distance of bags sold. 
Each fertilizer will have at least one pallet space. 


(a) Assuming that fractional numbers of pallet 
spaces are allowed, briefly explain how this pal- 
let allocation problem can be modeled by the 
multiobjective nonlinear program 


min 20/x; + 14/x2 + 9/x3 +5/x4 
min 200(%;/2) + 140(x; + x2/2) 
+ 90(x1 + x2 + x3/2) 
+ 50(x%1 +X. +3 + x4/2) 


4 
St Sox = 20 
j=l 
eae ly 
(b) [Q] and (c) & as in Exercise 8-1. 
8-5 Consider the multiobjective LP 


max x, +5x2 


Max Xj; 

s.t. X, — 2x, <2 
xy ot 2x7 <12 
2X1 + x%2<9 
As Xo 0 


(a) [| Show graphically that the optimal solu- 
tions taken separately for each objective do not 
coincide. 
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(b) [Q] Determine graphically whether each of the 
following is an efficient point: (2,0), (4,7), (3,3), 
(2,5), (2,2), (0,6). 

(c) [| Use graphic solution to sketch the efficient 
frontier for this model in an objective value 
graph like Figure 8.4. 


8-6 Do Exercise 8-5 for multiobjective LP 


min 5x1 — 9 

min x, + 4x 

s.t. —5x1 + 2x2 < 10 
xX, +X = 3 
xX, +2x, > 4 
X1,X2 = 0 


and points (4,0), (2,1), (3,3), (1,2), (5,0), (0,0). 
8-7 Consider the multiobjective LP 


max 6x, + 4x» 

max Xx 

s.t. 3x, + 2x2. < 12 
xX, + 2x2 < 10 
“eS 3 
X1,%2 20 


(a) [&] State and solve graphically a sequence of lin- 
ear programs to compute a preemptive solution 
giving priority to the first objective. Also verify 
that the result is an efficient point. 

(b) [9] State and solve graphically a sequence of lin- 
ear programs to compute a preemptive solution 
giving priority to the second objective. Also ver- 
ify that the result is an efficient point. 


8-8 Do Exercise 8-7 for the multiobjective LP 


min x; +X2 

min x, 

s.t. 2x, +X. > 4 
2x1 + 2x2 > 6 
Xi 4 
1,2 = 0 


8-9 Combine each of the following sets of objective 
functions into a single weighted-sum objective of the 
specified maximize or minimize form, weighting in- 
dividual objectives in the proportions indicated. 


(a) ©] min 3x, + 5x2 — 2x3 + 19x, 
max 17x2 —28x4 
min 34x, + 34x; 
Results form min, proportions 5:1:3. 
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(b) max 20x, — 4x. + 10x4 
Tx2 + 9x3 + 11x, 
min 23x, 
Results form max, proportions 2:1:1. 


min 


8-10 Return to the multiobjective linear program of 
Exercise 8-7. 


(a) EQ] Solve graphically with a weighted-sum ob- 
jective that weights the first objective twice as 
heavily as the second, and verify that the result 
is an efficient point. 

(b) [§] Solve graphically with a weighted-sum ob- 
jective that weights the second objective twelve 
times as heavily as the first, and verify that the 
result is an efficient point. 


8-11 Do Exercise 8-10 on the multiobjective linear 
program of Exercise 8-8. 


8-12 Convert each of the following multiobjec- 
tive optimization models to a goal program seeking 
the specified target levels, and minimizing the un- 
weighted sum of all goal violations. 


(a) Bl min 3x; + 5x. — x3 
max 11x. + 23x; 
s.t. 8x, + 5x2. + 3x3 < 40 
X2 —X3 <0 


X1,X2, x3 > 0 
Targets 20, 100. 
(b) max 17x, = 27x2 
min 90x. + 97x; 
s.t. xX, +X2 +x3 = 100 
40x, + 40x. — 20x3 > 8 
Xi{hs KiNG h = 0 
Targets 500, 5000. 


(c) Kl max 40x; + 23x, 
min 20x, — 20x, 
min 5x43 


s.t. xy +X) + 5x3 > 17 
40x, + 4x2 + 33x3 < 300 
X1,,%2,%3 = 0 


Targets 700, 25, 65. 


(d) min 12x; +34x2 + 7x3 
max Xx) —X3 
max 10x; +7x3 
s.t. 5x1 + 5x2 + 15x3 < 90 
xX, <19 


%1,%2,x3 = 0 
Targets 600, 20, 180. 


(e) re min 22x, + 8x2 + 13x3 
max 3x1 + 6x2 + 4x3 
S.t. 5x, + 4x. + 2x3 <6 
Xy + Xz +%3>1 
Make ks —= Vion 
Targets 20, 12. 
(f) max 4x, +In(x2) + x; + In(x3) 
min (x1)? + 9(%)? — x1x2 
s.t. XxX, +$X2+%x3 < 10 
4x +X%3> 6 
X1,X2,%3 >0 
Targets 20, 40. 


8-13 Consider the multiobjective LP 


max Xj, 

max 2x; + 2x) 

s.t. 2x, +x. <9 
x, <4 
x2 <7 
X1,X%2 >0 


(a) Sketch the feasible space and contours where 
the first objective equals 3 and the second equals 
14. 

(b) | Formulate a corresponding goal program 
seeking target levels 3 and 14 on the two ob- 
jectives and minimizing the unweighted sum of 
goal violations. 

(c) [Q] Explain why x = (2, 5) is optimal in the goal 
program of part (b) by reference to the plot of 
part (a). 

8-14 Do Exercise 8-13 on the multiobjective LP 


min x2 

max 5x, + 3x2 

St. 2x1 +3x,>6 
y< S) 
—X, +X. <2 
X1,X2 > 0 


using target levels 1 and 30 and solution x = (5, 2). 
8-15 Return to the multiobjective LP of Exercise 
8-13 with target levels 3 and 14 for the two objec- 
tives. 


(a) [Q] State and solve graphically [in (x1, x») space] 
a sequence of linear programs to compute a pre- 
emptive goal programming solution, prioritiz- 
ing objectives in the sequence given and using 
the given target levels. 

(b) [] Formulate a single goal program that imple- 
ments the preemptive goal programming pro- 
cess of part (a) by suitable deficiency variable 
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weighting, and demonstrate graphically that it 
produces the same solution. 

(c) [Q] Determine whether your solution in part (a) 
is an efficient point of the original multiobjective 
model. Could the outcome have been different 
if the target for the first objective were revised 
to 1? Explain. 


8-16 Return to the multiobjective LP of Exercise 
8-14. Do Exercise 8-15 on the model using target 
levels of 1 and 30 and a revised objective 1 target 
of 3. 


8-17 Return to the multiobjective District 88 water 
allocation problem of Exercise 8-1. 


(a) [] & State and solve with class optimization 
software a sequence of linear programs to com- 
pute a preemptive solution for the model of part 
(a) taking objectives in the order given. 

(b) &] Q State and solve with class optimization 
software a weighted-sum objective LP that 
weights the first objective twice as heavily as the 
second. 

(c) [Q] Suppose that district management sets a 
goal of at least 100 million acre-feet for irriga- 
tion and $144 million in income. Convert the 
model of Exercise 8-1(a) to a goal program seck- 
ing these target levels and minimizing the un- 
weighted sum of goal violations. 

(d) Use class optimization software and your goal 
program of part (c) to show that all goals can- 
not be fulfilled simultaneously by any feasible 
solution. 

(e) [Q] BSolve the model of part (c) with class op- 
timization software and compare to results for 
earlier methods. 

(f) [] & State and solve with class optimization 
software a sequence of linear programs to com- 
pute a preemptive goal programming solution 
for the model of part (c) taking objectives in the 
order given. 

(g) [Q] Formulate an alternative objective function 
in the goal program of part (c) that accomplishes 
the preemptive goal programming optimization 
of part (f) in a single step. 


8-18 Do Exercise 8-17 on the silicon wafer problem 
of Exercise 8-2 using target levels of 30,000 for cost 
and 13,000 for match. 

8-19 [6] Do Exercise 8-17 on the base closing prob- 
lem of Exercise 8-3 using target levels of 3% for 
readiness loss and 12,000 for displaced workers. 
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8-20 Do Exercise 8-17 on the pallet allocation prob- 
lem of Exercise 8-4 using target levels of 10 for refills 
and 2500 for walking. 


8-21 An architect is trying to decide what mix of 
single, double, and luxury rooms to include in a new 
hotel. Only $10 million is available for the project, 
and single rooms cost $40,000 to build, double rooms 
$60,000, and luxury rooms $120,000 each. Business 
travelers (in groups of one or more) average 0.7 of 
single-room rentals, 0.4 of double-room rentals, and 
0.9 of luxury room rentals. Family travelers occupy 
the remainder of each category. The architect would 
like her design to accommodate 100 total rooms for 
business travelers, but she would also like it to pro- 
vide 120 total rooms for family travelers. 


(a) [&] Formulate a weighted goal LP to decide how 
many of each room to include in the design. 
Weight the business traveler goal twice as heav- 
ily as the family traveler goal. 

(b) [6] QEnter and solve your goal program using 
class optimization software. 


8-22 Trustees of a major state university are trying 
to decide how much tuition* should be charged next 
year from four categories of full-time students: in- 
state undergraduates, in-state graduates, out-of-state 
undergraduates, and out-of-state graduates. The fol- 
lowing table shows present tuition levels (thousands 
of dollars) for each category, together with projected 
enrollments (thousands) for next year and estimates 
of the true university cost to educate students. 


Undergrad Grad 


In-state tuition 4 10 


Out-of-state tuition 12 15 
In-state enrollment 20 Ales} 
Out-of-state enrollment 10 4 


True cost 20 36 


New tuitions must raise $292 million, but trustees 
would also like in-state tuitions to recover at least 
25% of true cost and out-of-state, 50%. No tuition 
will be reduced from its present level, but the trustees 
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would also like to assure that no tuition category in- 
creases by more that 10%. 


(a) Formulate a goal LP to compute suitable tu- 
itions giving priority to the 10% increase goals 
Over cost recovery. 

(b) [| SEnter and solve your model with class op- 
timization software. 


8-23 A university library> must cut annual subscrip- 
tion expenses s; to some or all scientific journals 
j = 1,...,40 to absorb a $5000 per year budget 
cut. One consideration will be the sum of publishec 
counts c; of the number of times other journals cite 
papers in journal j, which is a measure of how sem- 
inal a journal is to research. Another is the sum o! 
usefulness ratings 7; (1 = low to 10 = high) solicited 
from university faculty. Finally, the library wants te 
consider the sum of ratings a; of the relative availabil- 
ity (1 = low to 8 = high) in nearby libraries, believing 
that journals readily available elsewhere need not be 
retained. 


(a) Formulate a multiobjective ILP to choose jour 
nals to drop. 

(b) Convert your multiobjective model to a goa 
program with targets at most C for total cita- 
tions, at most R for total faculty ratings, and at 
least A for total availability ratings. Weight all 
goals equally. 


8-24 The household sector is the largest consumer 0! 
energy in India,® requiring at least 10° kilowatthours 
per day in a large city such as Madras. This en- 
ergy can be obtained from sources j = 1 kerosene 
j = 2 biogas, j = 3 photovoltaic, 7 = 4 fuelwooe- 
generated electricity, 7 = 5 biogas-generated elec- 
tricity, 7 = 6 diesel-generated electricity, and j = ~ 
electricity from the national power grid, with effi- 
ciencies n; expressing the number of kilowatthours 
per unit of original fuel input of source j. Biogas is 
limited 1.3 x 10° units. Energy planners are look- 
ing for the best mix of these sources to meet the 10 
requirement in terms of a variety of conflicting ob- 
jectives. One is low total cost at p; per unit source 


“Based on A. G. Greenwood and L. J. Moore (1987), “An Inter-temporal Multi-goal Linear Program- 
ming Model for Optimizing University Tuition and Fee Structures,” Journal of the Operational Research 


Society, 38, 599-613. 


*Based on M. J. Schniederjans and R. Santhanam (1989), “A Zero—One Goal Programming Ap- 
proach for the Journal Selection and Cancellation Problem,” Computers and Operations Research, 16, 


557-565. 


®Based on R. Ramanathan and L. S. Ganesh (1995), “Energy Alternatives for Lighting in House- 
holds: An Evaluation Using and Integrated Goal Programming-AHP Model,” Energy, 20, 66-72. 


;. Another is to maximize local employment, which 
is estimated to grow at e; per unit of energy source 
. Finally, there are three types of pollution to be 
minimized: carbon oxides produces at c; per unit of 
source j, sulfides at s; per unit of j, and nitrogen ox- 
ides at nj per unit of j. 


a) Formulate a multiobjective LP to choose a best 
mix of sources. 

5) Revise your model as a goal program with maxi- 
mum targets P, C, S, and N for cost, carbon, sul- 
fur, and nitrogen, together with minimum target 
E for employment. 


8-25 The table that follows lists tasks like those re- 
guired of the U.S. Food and Drug Administration 
FDA) in carrying out its responsibilities to regulate 
drug laboratories, along with an importance weight- 
ing of each tasks and the estimated minimum number 
of staff hours required to accomplish the task fully. 


J Task Import Hours 
Laboratory quality 10 1500 

control program 

2 Analytical protocol 1 100 

3. Drug protocol 2 50 

4 Quality assurance 5 5D 
guidelines 

5 Safety programs 10 135 

6 Stability of testing 5 490 

7 Calibration of 4 2000 
equipment 

Total 4330 


Unfortunately only 3600 of the 4330 total hours are 
available to execute all tasks. FDA management 
seeks an allocation of the 3600 hours that addresses 
the goal hours in the table on the basis of minimizing 
total importance of undersatisfaction. Formulate a 
zoal LP to determine the most appropriate alloca- 
tion. 

8-26 The Lake Lucky dam’ retains water in the 
reservoir lake of the same name to prevent flood- 
ing downstream and to assure a steady year-round 
supply of water for wildlife and nearby cities. The 
dest available projections call for inflows i, (cubic 
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meters of water) to arrive at the reservoir on up- 
coming dayst = 1,..., 120. A minimum of at least r 
must be released through the dam every day to main- 
tain downstream water quality, but the daily release 
can go as high as 7. The current storage in the reser- 
voir is Sy cubic meters, and storage levels must be 
kept between minimum and maximum limits s and5 
throughout the 120-day period. Reservoir managers 
seek a 120-day plan that releases, as close as possi- 
ble to target, R cubic meters each day and maintains 
a target level of § cubic meters stored in the reser- 
voir. Both below- and above-target outcomes are 
considered equally bad, and release deviations are 
weighted equally with those for storage. Formulate 
a goal LP to compute a reservoir operating plan. As- 
sume that no water is lost (i.e. all water that enters 
the reservoir eventually flows over the dam). 


8-27 A retail store has hired a marketing consultant 
to help it decide how to evaluate the quality® of its 
service. The store would like to evaluate customer 
perceptions about i = 1, employee attitude; 7 = 2, 
employee competence; i = 3, product quality; and 
i = 4, sensory (look, sound, smell) appeal of the 
store. The consultant is considering 6 survey instru- 
ments for the task: 7 = 1, point-of-purchase cards 
measuring employee service delivery; j = 2, point- 
of-purchase cards measuring product quality; j = 3, 
open-ended comment cards; j = 4, a focus group of 
known customers; and j = 5, a telephone survey of 
known customers. The instruments are not equally 
useful in measuring the desired quality attributes, so 
the consultant has developed (1 = poor to 10 = ex- 
cellent) ratings 7;; scoring the value of instrument 
j in assessing attribute i. Instruments j would cost 
c; dollars and require h; hours of store employee 
time. Ideally, the instruments chosen should total 
at least 30 points on each rated attribute, but their 
total cost must not exceed $10,000, and the total em- 
ployee hours invested must not be more than 500. 
Formulate a goal ILP to choose a suitable combina- 
tion of instruments. 


8-28 A steel workpiece is to be cut? in a lathe to a 
target depth of 0.04 inch. Empirical studies with this 


7Based on K. K. Reznicek, S. P. Simonovic, and C. R. Bector (1991), “Optimization of Short-Term 
Jperations of a Single Multipurpose Reservoir—A Goal Programming Approach,” Canadian Journal of 


-ivil Engineering, 18, 397-406. 


8Based on M. J. Schniederjans and C. M. Karuppan (1995), “Designing a Quality Control System 
aa Service Organization: A Goal Programming Case Study,” European Journal of Operational Research 


*2, 249-258. 


*Based on R. M. Sundaram (1978), “An Application of Goal Programming Technique in Metal 
Cutting,” International Journal of Production Research, 16, 375-382. 
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type of steel express the properties of the resulting 
product as 


finish =) Oral yo F307 
power A Wied 
time A vf 


where v is the cutting speed in revolutions per 
minute, fis the feed rate in inches per revolution, and 
d is the depth of cut in inches. Finish variation must 
be kept no greater than 150, required power no more 
that 4.0, cutting speed between 285 and 680, and feed 
rate between 0.0075 and 0.0104. Within these limits, 
the first priority goal is to make the depth of cut as 
close as possible to 0.04 inch, and the second is to 
complete the cut in at most time 1.5 (minutes). 


(a) Formulate a preemptive goal NLP to choose 
cutting parameters. 

(b) &] SEnter and (at least locally) solve your goal 
program using class optimization software. 


8-29 Trees in available wild forest!® stands j = 
1,...,300 have been measured in terms of desirable 
traits i = 1,...,12 such as rate of growth, disease 
resistance, and wood density. Computations of aver- 
ages and standard deviations in traits over the stands 
then produced numbers z;; of standard deviations 
that stand j falls above or below the average for trait i. 
Corresponding scores for the offspring of a planned 
breeding program may be assumed to combine lin- 
early (i.e. the result will be a sum of scores for the 
wild stands used, weighted by the proportion each 
stand makes up of the breeding population). 


(a) Formulate a multiobjective LP to choose a 
breeding population that maximizes the off- 
spring standard deviation score of each trait. 

(b) Convert your model to a goal program with a 
target of breeding offspring 2 standard devia- 
tions above the mean on all traits. Weight the 
undersatisfaction of each goal equally. 


8-30 Soar, a retailer catering to upscale suburban- 
ites," wishes to be represented in all the major malls 


Objective Linear Programming,” Interfaces, 14:5, 96-104. 


Journal of Operational Research, 73, 17-22. 


New Product Formulation,” Interfaces, 19:5, 49-56. 


°Based on T. H. Mattheiss and S. B. Land (1984), “A Tree Breeding Strategy Based on Multiple 


'Based on R. Khorramshahgol and A. A. Okoruwa (1994), “A Goal Programming Approach to 
Investment Decisions: A Case Study of Fund Allocation Among Different Shopping Malls,” European 


Based on J. S. Schmidt and L. C. Meile (1989), “Taguchi Designs and Linear Programming Speed 


3Based on T. J. Stewart (1988), “Experience with Prototype Multicriteria Decision Support Systems 
for Pelagic Fish Quota Determination,” Naval Research Logistics, 35,719-731. 
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j =1,...,5 in the Atlanta region. However, Sos: 
wants to divide its investment budget b in propor 


tion to three major measures of mall attractiveness 
weekly patronage, p;; average patron annual income 
aj; and number of large anchor stores, s;. That * 
they would like ratios of investment inj to p; all * 
be equal, and similarly for corresponding ratios t 


the a; and s;. Formulate a goal LP model to choos: 
an appropriate allocation, weighting all deviation: 


equally. 


8-31 Shipley Company” is designing a new photore- 


sist, which is a chemical coating used in the phot 

engraving of silicon chips. Important characteristic 
of photoresists are 7 = 1, high flow (liqueficatior 
temperature; 7 = 2, low minimum line-space resolu 
tion; 7 = 3, retention of unexposed areas when & 
posed circuits are developed away at least b,; i = = 
photospeed (minimum exposure time to create a= 
image) between b, and by; and i = 5, exposure ex 
ergy requirements between b, and bs. Ingrediens 
j = 1,...,24 produce highly nonlinear effects os 
these five important characteristics. However, de 


signed experiments have demonstrated that the - 


fects can be assumed linear within ingredient quar 
tity ranges [€;, uj] under consideration. Betwees 
these limits, each additional unit of ingredient j adc 
a;; to characteristic 7 of the photoresist. 


(a) Formulate a 2-objective LP model to choose th: 
best composition for both flow temperature an- 
line-space resolution. 

(b) Explain the meaning of an efficient point sole 
tion to your model in part (a), and describe ho» 
one could be computed. 


8-32 South African wildlife management officials 
must decide annual quotas (in thousands of tons) f= 
harvesting of pilchards, anchovies, and other fish 
the pelagic family in the rich waters to the west 
the Cape of Good Hope. One consideration is th: 
maximization of fishing industry income, which is & 
timated at 110 rand/ton for pilchards, 30 rand/ton f= 
anchovies, and 100 rand/ton for other pelagic. St 


the harvest should leave the year-end populations of 
the 3 types of fish as large as possible. Scientists es- 
umate that initial biomasses of the 3 types are 140 
thousand tons, 1,750 thousand tons, and 500 thou- 
sand tons, respectively. Due to differences in breed- 
ing characteristics, each thousand tons of pilchards 
harvested will deplete the population by 0.75 thou- 
sand tons, each of anchovies by 1.2 thousand tons, 
and each of other pilchards by 1.5 thousand tons. A 
Snal consideration is maintaining the ocean ecosys- 
tem. Important measures are the number of breed- 
ing pairs of penguins (thousands), which is estimated 
as 70 + 0.60 (ending biomass of pilchards), and the 
number of such pairs of gannets (thousands), which 
is estimated at 5+0.20 (ending biomass of pilchards) 
— 0.20 (ending biomass of anchovies). 


a) Formulate a multiobjective LP to maximize all 
measures of success using the decision variables 
(i = 1, pilchards; i = 2, anchovies; i = 3, other 
pelagic) 


x= quota for fish type i (thousand tons) 


Vi 2 ending biomass of fish type i 
(thousand tons) 


b) Revise your formulation as a goal program with 
minimum targets for the 6 objectives being 38 
million rands income, ending biomasses of 100, 
1150 and 250 thousand tons, and ending breed- 
ing populations of 130 and 25 thousand pairs. 
Give preemptive priority to maintaining the 
populations of the 3 types of fish. 

[6] &] Enter and solve your goal program ver- 
sion using class optimization software. 


fo) 
wa 


8-33 The state of Calizona will soon be opening cen- 

ers!* in 12 of its county seats i = 1,..., 100 to pro- 
mote solid waste recycling programs. Each selected 
site will service a district of surrounding counties. 
The state wants to minimize the sum of distances dj; 
populations p; in counties i must travel to their dis- 
trict center at 7. But they would also like to come 
as close as possible (in total deviation) to having 3 
centers in each of the 4 Environment Department 
regions. Counties i = 1,...,12 make up region 1, 
i=13,...,47 region 2, i = 48, ..., 89 region 3, and 
i= 90,..., 100 region 4. 
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(a) Formulate a 2-objective ILP model of this fa- 
cility location problem using the main decision 
variables (i,j = 1,..., 100) 


1 if county 7 is served by a center 
ie = al] 
0 otherwise 
ok 1 if county j gets a center 
j — 


0 otherwise 


(b) Explain the meaning of an efficient point solu- 
tion to your model of part (a) and describe how 
one could be computed. 


8-34 Every year the U.S. Navy must plan the reas- 
signment of thousands of sailors finishing one tour of 
duty in specialties i = 1,...,300 and preparing for 
their next one, k. The reassignment may also require 
a move of the sailor and his or her dependents from 
present base location j = 1,...,25 to a new base &. 
To plan the move, personnel staff estimate numbers 
s;; Of sailors at base j in specialty i who are ready 
for reassignment, d;,- of sailors needed at base ¢ in 
specialty &, as well as average costs cj, of moving a 
sailor and dependents from base j to base £. Train- 
ing is required for the new position if it involves a 
different specialty than the present one, and capac- 
ity limits uv, at training schools limit the number that 
can be trained for any specialty & during the year. 
One Navy objective in planning reassignment is to 
Keep total relocation costs as low as possible. How- 
ever, other considerations arise from the fact that 
full staffing levels d,. can almost never all be met. 
Naturally, the Atlantic fleet (bases j,é = 1,..., 15) 
wishes to maximize the fraction of its slots actually 
filled, and the Pacific fleet (bases j,€ = 16,...,25 
prefers to maximize the fraction of its needs accom- 
modated. Formulate a 3-objective LP model to aid 
the Navy in planning reassignments using the deci- 
sion variables 


Xi.x,¢ & number of sailors presently in specialty i at 
base j reassigned to specialty k at location @ 


8-35 The sales manager of an office systems dis- 
tributor> has sales representatives i = 1,...,18 to 
assign to both current accounts j = 1,...,250 and 


14Based on R. A. Gerrard and R. L. Church (1994), “Analyzing Tradeoffs between Zonal Constraints 
and Accessibility in Facility Location,” Computers and Operations Research, 21, 79-99. 

1S Based on A. Stam, E. A. Joachimsthaler, and L. A. Gardiner (1992), “Interactive Multiple Objec- 
tive Decision Support for Sales Force Sizing and Deployment,” Decision Sciences, 23, 445-466. 
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the development of new accounts. Each current ac- 
count will be assigned at most one representative. 
From prior experience, the manager can estimate 
monotone-increasing nonlinear functions 


rij Wij) & current-period sales revenue that will 
result from allocating w;; hours of 
representative / time to account j 


aj (x;) 2 present value of future sales revenue 
that will derive from assigning 
representative 7 to x; hours of new 
account development 


Each representative has 200 hours to divide between 
his or her assignments during the planning period, 
but the manager would like to maximize both total 
current revenue and total new account value. 


SUGGESTED READING 


Goicoechea, Ambrose, Don R. Hansen, and Lucien 
Duckstein (1982), Multiobjective Decision Analysis 
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(a) Formulate a 2-objective INLP to determine = 
allocation plan using the decision variables (i = 
Ibe. ASS) SA e250) 


A : a 
wi; = hours spent by representative i on 
account j 


A + 
x; = hours spent by representative i on new 
account development 


il if representative 7 is assigned 
‘A : 
yij = account / 
0  — otherwise 
(b) Explain the meaning of an efficient point sol 


tion to your model of part (a) and describe how 
one could be computed. 


with Engineering and Business Applications, Wile 
New York. 
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Shortest Paths 
and Discrete 
Dynamic Programming 


From our earliest discussions in Chapters 1 and 2, we have seen a tradeoff between 
generality of operations research models and tractability of their analysis. The more 
specialized the model form and the longer the list of underlying assumptions, the 
richer and more efficient the analysis that is possible. 

In this chapter we extend that insight to a new extreme by introducing the most 
specialized and thus most efficiently solved of all broad classes of optimization mod- 
els: shortest paths and discrete dynamic programs. Dynamic programming methods 
proceed by viewing the problem or problems we really want to solve as members of 
a family of closely related optimizations. When model forms are specialized enough 
to admit just the right sort of family, strong connections among optimal solutions to 
the various members can be exploited to organize a very efficient search. 


9, SHORTEST PATH MODELS 


Whether in urban traffic, or college hallways, or satellite communications, or the 
surface of a microchip, it makes sense to follow the shortest route. Such shortest 
path problems are the first we will attack. 


EXAMPLE 9.1: LITTLEVILLE 


As usual, it will help to begin with an example setting. Suppose that you are the city 
traffic engineer for the town of Littleville. Figure 9.1(a) depicts the arrangement of 
one- and two-way streets in a proposed improvement plan for Littleville’s downtown, 
including the estimated average time in seconds that a car will require to transit each 
block. 

From survey and other data we can estimate how many driver trips originate 
at various origin points in the town, and the destination for which each trip is bound. 
But such survey data cannot indicate what streets will be selected by motorists to 
move from origin to destination in a street network that does not yet exist. 

One of the tasks of a traffic engineer is to project the route that drivers will elect, 
so that city leaders can evaluate whether flows will concentrate where they hope. A 


409 


410 Chapter Nine e Shortest Paths and Discrete Dynamic Programming 


(a) Proposed street network 


38 


(b) Corresponding graph 


FIGURE 9.1 Littleville Shortest Path Example 


good starting point is to assume that drivers will do the most rational thing—follos 
the shortest time path from their origin to their destination. We need to compute =” 
such shortest paths. 


Nodes, Arcs, Edges, and Graphs 


Figure 9.1(b) shows the first step. We abstract the given flow pattern—here a stre=” 
system—into a graph or network. 
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These are not the newspaper sort of graphs comparing bars, or zigzag lines, or 
piles of little coins. 


In Figure 9.1(b) nodes represent intersections. For convenience we have num- 
bered the nodes 1 to 10. One-way streets between intersections of the Littleville 
network yield arcs in our graph; two-way streets produce edges. 

We denote arcs or edges by naming their terminal nodes. Thus (5, 1) and (3, 4) 
are arcs in Figure 9,1(b), while (7, 9) and (5, 6) are edges. Order matters for arcs, so 
that it would be incorrect, for example, to call the arc from node 10 to node 8 arc 
(8, 10). It must be (10, 8). There is more flexibility in referencing edges. An edge 
between nodes i and j could be called either (i, /), or (j, i), or both, although it is more 
common to name the smallest node number first. For example, we could refer to the 
edge between nodes 2 and 6 of Figure 9.1(b) as either (2, 6) or (6, 2), but (2, 6) would 
be more standard. 


Consider the following graph: 


@ w) 


Identify its nodes, arcs, and edges. 


Modeling: The node set of this graphis V A {1, 2, 3, 4,5}. Arcs form.A 4 {(2, 4), (3, 4), 
(4, 3), (5, 3)}. Edges are FE A {(1, 2), (1, 3), (4, 5)}. 


Paths 


Our interest in this chapter is paths. 
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Two of several paths from node 3 to node 8 in the Littleville example are illustrated in 
Figure 9.2(a). One proceeds 3-7-10-8, using edges (3, 7) and (7, 10), plus arc (10, 8 
Another is 3-4-10-8, following arc sequence (3, 4), (4, 10), (10, 8). The pattern 3—7- 
6-5-8 of Figure 9.2(b) is not a path because it transits arc (8, 5) in the wrong direction. 
Sequence 3—7-6—-9-7-10-8 also fails the definition of a path; it repeats node 7. 


(b) Sequences from 3 to 8 that are not paths 


FIGURE 9.2 Paths of the Littleville Example 


Rear to the Seah of Sample Fentike O: it Identify all atts Fort node 1 % 
node 5. 


Modeling: The two available paths are 1-2-4-5 and 1-3-4-5. Seer 1-3-5 isnot 
a path because it violates direction on arc (3, 5) 
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Shortest Path Problems 


When an application includes costs or lengths on the arcs and edges of a graph, we 
are confronted with an optimization. 


9.5 | Shortest path problems seek minimum 1 total length paths between Spec: 
‘ified pairs of nodes i ina fei | — 


In the Littleville example of Figure 9.1, lengths are travel times in seconds. The 
first path from node 3 to node 8 shown in Figure 9.2(a) totals (30 + 49 + 28) = 107 
seconds in length. The shortest path from 3 to 8 is the second 3-4-10-8 sequence 
with length (13 + 38 + 28) = 79 seconds. 


Classification of Shortest Path Models 


Shortest path problems arise both as main decision questions and as steps in other 
computations. There are many variations, depending on the type of network and 
costs involved, and the pairs of nodes for which we need solutions. 

We will see in the sections to come that shortest path algorithms are highly 
specialized to exploit particular properties. Thus it is important to distinguish cases. 

Our Littleville example illustrates one combination. The graph has both arcs 
and edges, all link lengths are nonnegative, and our traffic engineering task requires 
shortest paths between all pairs of nodes. To summarize: 


e Name: Littleville 
e Graph: arcs and edges 
: e Costs: nonnegative 
. e Output: shortest paths 
e Pairs: all nodes to all others 


Before passing to algorithms, we introduce some examples illustrating other possi- 
bilities. 
| EXAMPLE 9.2: TEXAS TRANSFER 


Figure 9.3 displays a map of highway links between several major cities in Texas. 
Numbers on the edges in Figure 9.3 show standard driving distance in miles, 

Texas Transfer, a major trucker in the southwest, ships goods from its hub 
warehouse in Ft. Worth to all the cities shown. Trucks leave the hub and proceed 
directly to their destination city, with no intermediate dropoffs or pickups. 

Texas Transfer drivers are allowed to choose their own route from Ft. Worth 
to their destination. However, management’s proposal in current labor negotiations 
calls for drivers to be paid on the basis of shortest standard mileage to the location. 
That is, they will be paid according to the length of the shortest path from Ft. Worth to 
their destination city in the network of Figure 9.3. To see the impact of this preposal, 
we need to compute such shortest path distances for all cities. 

Notice that the character of this shortest path task differs significantly from 
the Littleville example. Here we need only optimal path lengths, not the paths 
themselves. Also, we need shortest path lengths for only one origin or source—the 
Ft. Worth hub. 
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El Paso 


Corpus Christi 


FIGURE 9.3. Network for Texas Transfer Example 


e Name: Texas Transfer 

e Graph: edges only 

e Costs: nonnegative 

e Output: shortest path lengths 
e Pairs: one node to all others 


EXAMPLE 9.3: TWO RING CIRCUS 


The Two Ring Circus is nearing the end of its season and planning a return to winte= 
headquarters near Tallahassee, Florida. Present commitments will end in Lincol= 
Nebraska, but there are still some opportunities for bookings in cities along the rout: 
home. 

Figure 9.4(a) shows the travel routes available and the estimated costs (in thou. 
sands of dollars) of moving the circus over those routes. It also designates the cities: 
where bookings have been offered and the anticipated net receipts (in thousands & 
dollars). 

We want to compute the optimal path home for Two Ring. Notice that the cos 
of a path is the difference of travel costs and net receipts from performances along 
the way. But receipts occur at nodes of the network, not on edges as shortest pat® 
models require. 


Undirected and Directed Graphs (Digraphs) 


The Two Ring network of Figure 9.4(a) is an undirected graph because it has on! 
edges (undirected links). The key to incorporating receipts at nodes is first to conve= 
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Davenport (1.6) 


Lincoln (0.0) 


(a) Original travel network 
Davenport (1.6) 


(b) Corresponding digraph 
FIGURE 9.4 Two Ring Circus Example Graphs 


to an equivalent directed graph or digraph, that is, a graph having only arcs (directed 
links). 


Such a directing of a graph is easy: 
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Davenport 


(c) After incorporating node values 


FIGURE 9.4 Two Ring Circus Example Graphs (Continued) 


The two parallel arcs replacing each edge merely make explicit the two directional 
options for using the edge in a path. Whether directed or not, a path will use the link 
no more than once because paths cannot repeat nodes. 


SAMPLE EXERCISE 9.3: DIRECTING A GRAPH 


Consider the following graph: 


© % @ 


Show the equivalent digraph. 


Modeling: Applying [9.6], we replace all edges of the given graph with a pair of 
opposed arcs with the same length as the edge. The result is 


Notice that arcs are unchanged and that the two arcs replacing each edge both have 
the same length as the edge. 
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Two Ring Example Model 


Figure 9.4(b) shows the result of directing the Two Ring network. For example, 
Wichita-to-Little Rock edge (3, 5) becomes arcs (3, 5) and (5, 3), both with the same 
cost as the original edge. 

Net receipt values remain on nodes in part (b) of the figure. Part (c) moves 
them to arcs by subtracting net receipts in each city from the cost of all outbound 
arcs there. For example, Little Rock-to-Jackson arc (5, 7) now has cost 


2.1 —3.0 = —0.9 


to account for net receipts if Two Ring plays Little Rock. Reverse arc (7,5) from 
Jackson to Little Rock has cost 


Pe ee 1 0 leh 


to include Jackson receipts. If either of these arcs is used in an optimal path, the full 
effect of both travel costs and show revenues will be reflected. 

It may seem wrong to subtract receipts from all outbound arcs from a city when 
show receipts can be realized only once. Remember, however, that a path can visit 
any node only once—entering on an inbound arc and leaving on an outbound. It 
follows that receipts for a town can be part of any path’s length at most once. 

The Two Ring digraph of Figure 9.4(c) also introduces another class of shortest 
path models. This time arc costs, being net dollar amounts, have unpredictable sign. 
Also, we require a shortest path for only one pair of nodes: Lincoln to Tallahassee. 


e Name: Two Ring Circus 

e Graph: directed 

e Costs: arbitrary 

e Qutput: shortest path 

e Pairs: one source to one destination 


9,2 DYNAMIC PROGRAMMING APPROACH TO SHORTEST PATHS 
SE EE AE 


Dynamic programming methods exploit the fact that it is sometimes easiest to solve 
one optimization problem by taking on an entire family. If strong enough relation- 
ships can be found among optimal solutions to the various members of the family, 
attacking all together can be the most efficient way of solving the one(s) we really 
care about. 


Families of Shortest Path Models 


We will see later in the chapter how we sometimes have to invent a family of prob- 
lems to solve a particular one by dynamic programming. With shortest path mod- 
els the appropriate family is already at hand. We need only exploit the fact that 
most applications require optimal paths or path lengths for more than one pair of 
nodes. 


xploit Te 
On nodes. - 


path lengths) for different pai 
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The Littleville and Texas Transfer examples of Section 9.1 already seek paths 
for multiple pairs of nodes. Littleville requires an optimal path between every pair of 
nodes, and Texas Transfer needs optimal path lengths from one node to every other. 

Even if only one optimal path is required, our dynamic programming approach 
to shortest path problems embeds the case of application interest in such a family of 
problems for different origin—destination pairs. In particular, we will solve the Two 
Ring circus example, which needs an optimal path for only one origin-destination 
pair, by embedding it in the problem of finding shortest paths from the origin to all 
other nodes. é 


Functional Notation 


With many optimizations going on at once, we will need some notation to keep score. 
Specifically, we denote optimal solutions and solution values as functions of the family 
member. Square brackets [...] enclose the parameters in such functional notation. 

The ‘Texas Transfer and Two Ring examples require optima for one source to 
all other nodes. Corresponding functional notation is 


v[k] A length of a shortest path from the source node to node k 
(= +00 if no path exists) 
1 if arc/edge (i, /) is part of the optimal 
BH tial (ee path from the source node to node k 
0 otherwise 


Notice that we adopt the convention that v[k] = +oo when there is no path to k. 
Examples such as Littleville, which need optimal paths from all nodes to all 
others, have two parameters: 


v[k, £] A length of a shortest path from node k to node £ 
(= +00 if no path exists) 


xij[k, €] S | 


Again v[k, ¢] = +00 indicates that no path exists from k to @. 


1 if arc/edge (i, j) is part of the optimal path from node k to node ¢ 
0 otherwise 


Consider the problem of finding the shortest path from source node 1 in the following 
graph to all other nodes. 


(a) Use inspection to determine all required shortest paths. 


(b) Detail optimal solutions and solution values in functional notation. 
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Modeling: 


(a) It is easy to check that optimal paths to nodes 2, 3, and 4 are 1-2, 1-2-3, and 
1-2-4, respectively. There is no path from source 1 to node 5. 


(b) In functional notation these optimal paths imply that 


v[1] = *1,2[1] = x1,3[1] = x2 3[1] = x24[1] = x3.4[1] = 0 
v[2] = ¥1,2[2] = 1, x1,3[2] = x2 3[2] = %2,4[2] = x3,4[2] = 0 
v[3] = 11 %1,2[3] = ¥2,3[3] = 1, x1,3[3] = x2.4[3] = x3,4[3] = 0 
v[4] =9 *1,2[4] = x2,4[4] = 1, x13[4] = x23[4] = x3,4[4] = 0 
v[5] = +00 (no path) 


Optimal Paths and Subpaths 


To formulate a dynamic programming approach to our shortest path models, we 
must identify connections among optimal solutions to problems for different pairs 
of nodes. How is the optimal path for one pair of nodes related to the optimal path 
for another? 

To begin to see, examine Figure 9.5. The highlighted path through Austin and 
San Antonio is the shortest from the origin at Ft. Worth to Corpus Christi. Think 
now about San Antonio. The figure’s Ft. Worth—Austin-San Antonio path is one way 
to get to San Antonio, but could any other path be shorter? Certainly not. If there 
were a better way to get from Ft. Worth to San Antonio than the path indicated in 


Amarillo 


El Paso (9) Houston 


San Antonio 


Corpus Christi (10, 


FIGURE 9.5 Optimal Path of the Texas Transfer Example 
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Figure 9.5, that better route would also yield an improvement on the Corpus Christi 
optimum. We need only follow it to San Antonio and finish with the link from San 
Antonio to Corpus Christi. 


Negative Dicycles Exception 


It seems safe to conclude from examples like Figure 9.5 that optimal paths must 
always have optimal subpaths. Unfortunately, there is a glitch. 

Consider the example in Figure 9.6. The shortest path from source s = 1 to 
node 3 clearly proceeds 1-2-3 with v[3] = 5. Still, the subpath 1-2 is not optimal. 
Sequence 1-3-4—2 has lesser length v[2] = —3. 

The logic of the preceding subsection suggests that there must be a contradic- 
tion. Simply extending the optimal path to node 2 with arc (2, 3) should improve the 
optimum at node 3. But that extension produces the sequence 1-3-4—2-3, which is 
not a path because it repeats node 3. Eliminating the repetition gives path 1-3, but 
its length 10 is worse than 1-2-3. 

The troublemaker is a negative dicycle. 


9.8 A dicycle is a path that begins and ends at the same node, and a negative _ 
dicycle is a dicycle of negative total length. 


Figure 9.6 contains negative dicycle 3-4-2-3 with length 
I (25) 3 = — 10 


Its presence permits nonoptimal subpath 1-2 because extending shortest 1 to 2 path 
1-3—-4—2 with arc (2, 3) closes a dicycle that cannot be eliminated without increasing 
the length of the route it offers to node 3. 

Negative dicycles do not occur often in applied shortest path models because 
they imply a sort of economic perpetual motion. Each time we transit a negative di- 
cycle we end up closer (in total length) to any source that we were when we started. 
However, when negative dicycles are present, shortest path models become so dra- 
matically more difficult that our whole solution approach must change. 
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FIGURE 9.6 Example with a Negative 


vi4] = 17 Dicycle 


Modeling: The dicycles of this example are 1-2-3-4-1 of length —2, and 1-3-4-1 
of length 14. Only the first is a negative dicycle. Sequence 1-2-3-1 is not a dicycle 
because it violates direction. 


Principle of Optimality 

Fortunately, negative dicycles are the only bad case in shortest path analysis. Just as 
with the instance of Figure 9.6, a subpath of a shortest path can fail to be optimal 
only if a negative dicycle is present. Other forms satisfy the simple insight known as 
the principle of optimality for shortest path models: 


Functional Equations 


To see how principle | 9.10 | helps us to compute optimal paths, let us limit our atten- 
tion to one node to all other circumstances of models such as that of Texas Transfer 
and the Two Ring Circus. If optimal paths must have optimal subpaths, it follows 
that a shortest path can be constructed by extending known shortest subpaths. 


Functional equations detail such recursive relationships. 


Functional Equations for One Node to All Others 


In one node to all other shortest path cases, functional equations relate shortest path 
lengths v[k]. 


The minimization in the second expression is taken over all neighbors i leading to k 
in the graph. If none exist, we adopt the convention that min{nothing} 2 + oo. 
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The first part of the functional equations in| 9.12 | simply says that the length of 
the shortest path from s to itself should = 0. The rest of the functional equations fix 
the recursive relationships implied by principle | 9.10}. In words, they say that the 
shortest path length for node j must be the best single arc/edge extension of optimal 


paths to neighboring nodes i. 
For an example, return again to San Antonio node j = 7 in Texas Transfer 


Figure 9.5. The corresponding equation is 
v[7] = min{v[6] + c6,7, v[8] + ¢g,7, v[9] + ¢9,7, v[10] + c10,7} 
min{v[6] + 213, v[8] + 79, v[9] + 199, v[10] + 153} 


That is, the length of the shortest path to San Antonio node 7 must be the least of 
l-edge extensions to optimal paths for neighboring nodes i = 6 (San Angelo), 8 
(Austin), 9 (Houston), and 10 (Corpus Christi). One of those must constitute an 
optimal subpath of the shortest path from Ft. Worth to San Antonio. 


SAMPLE EXERCISE 9.6: UNDERSTANDING FUNCTIONAL EQUATIONS 


Return to Sample Exercise 9.4. Write all corresponding functional euittions ane 
verify that they are satisfied by shortest path lengths computed in Sample Exercise 
9.4(b). 


Analysis: Following [9.12], functional equations are 
v[1] = 0 
v2] = min{v[1] + c1,2, v[3] + ¢3,2} 
v[3] = min{v[1] + c13, v[2] + c2,3, v[4] + cas} 
v[4] = min{v[2] + c2,4, v[3] + c3,4} 
v[5] = min{} 
Now substituting optimal values yields 
DL 0 
v[2] = min{0 + 3, 114 8} =3 
y[3] = minf0+ 15,3+8,9+5}=11 
v[4] = min{3 + 6,11+5}=9 


v[5] = min{} = +00 


Sufficiency of Functional Equations in the One to All Case 

In the absence of negative dicycles, shortest path lengths satisfy functional equations 
because they satisfy principle of optimality . Optimal paths must extend 
optimal subpaths. 


The algorithms that make up most of this chapter depend on the fact that the 
result works in both directions. 


9.2 Dynamic Programming Approach to Shortest Paths 


9.13 | Node values vf&] 
_est paths from a give 


That is, we can compute shortest path lengths v[k] simply by computing values that 
satisfy the functional equations. 

To see why, examine the v[k] values of Texas Transport Figure 9.7. Certainly, 
the value a Ft. Worth source node s = 3 is correct. The length of a shortest path 
from Ft. Worth to Ft. Worth must = 0. 


Ft. Worth 


Lubbock ci 
v[2] = 347, uEl-8 
¢ 


Abilene & 
v [5] = 180 


git 


Austin 
v [8] = 195 


San Angelo 


El Paso v [6] =272 Houston 
v[4] = 623 HENS aos 
San Antonio 
v [7] =274 


Corpus Christi 
v [10] = 427 


FIGURE 9.7 Sufficiency of Functional Equations 


For other nodes, say El Paso node 4, we make two observations. First, functional 
equations | 9.12 | assure that v[4] is the length of some path from s to node 4. This is 
true because there must be at least one neighbor i of node 4 that has 

[4] = v[i] + cig 
Here i = 5 (Abilene) works because it achieves the minimum in 9.12). But Abilene 
node 5 must, in turn, have a neighbor j with 

v[5] = v[j] + es 


Here it is] = 3. We continue in this way until source node s is reached (as it just 
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was). Then summing the relationships, simplifying, and using v[s] = 0 produces 
v[4] + v[5] = v[5] + vis] + ¢s,5 + ¢5,4 
[4] = vis] +65 + es 
v[4] = ¢s,5 + ¢5,4 


That is, v[4] is the length of path s — 5 — 4. 

The other half of the argument is to show that no other path can have shorter 
length than the one associated with v[4]. Consider the s-1-2-4 path highlighted with 
dashed lines in Figure 9.7. Because all values satisfy functional equations 


v[1] < vfs] + ¢s1 
v2] < v[1] + ¢1,2 
v[4] < v[2] + c24 
Again summing, simplifying, and using v[s] = 0, we obtain 
v[1] + v[2] + v[4] < vfs] + v[1] + v[2] + ¢s1 + c12 + €2,4 
v[4] < vfs] + ¢s,1 + ¢1,2 + ¢2,4 
v[4] < ¢s1 +12 + €2,4 


The path indicated cannot have length less than v[4]. 


SAMPLE EXERCISE 9.7: VERIFYING SUFFICIENCY OF FUNCTIONAL EQUATIONS 


Consider the following graph and associated node values v[k]: 


v [2] =3 


v[3] =5 


(a) Verify that given values v[k] satisfy functional equations : 
(b) Identify the path associated with the v[A] of each node k. 
(c) Use functional equations to show why the length of path 1-3-2-4 must be > v[4]. 


Analysis: 

(a) v[1] = 0 
vy[2] = min{v[1] + c12, v[3] + 3,2} = min{0+ 3,5+9} =3 
v[3] = min{v[1] + c13, v[2] + co.3} = minf0+ 5,3 +9} =5 
v[4] = min{v[2] + c24, v[3] + c3,.4} = min{3 + 8,5+15}=11 


9.2 Dynamic Programming Approach to Shortest Paths 


5) Label v[s] = 0 is the length of a null path from s = 1 to itself. For k — 2, we look 

‘or the neighbor achieving the minimum in the functional equation of v[2]. Here it 
ss = 1, so the path is 1-2 with length v[2] = 3. For k = 3, similar thinking yields 
path 1-3 with length v[3] = 5S. Node k = 4 takes two steps. At node 4 the neighbor 
achieving the minimum is i = 2. Its minimum, in turn, occurred for i’ = 1. Thus the 
path is 1-2-4 with length v[4] = 11. 


¢) Functional equations for nodes 4, 2, and 3 imply that 
v[4] < v[2] + c2,4 
v[2] < v[3] + ¢3.2 
v[3] < v[1] +¢3,1 
Summing, simplifying, and using v[1] = 0, we have 
v4] + v[2] + [3] < v[1] + v[2] +, [3] +13 +32 +024 
v[4] < v[1]4+c13 +032 +er4 
v[4].< 13. + ¢3,2 + C24 


Functional Equations for All Nodes to All Others 

For shortest path problems requiring optimal paths between all pairs of nodes (Lit- 
‘leville is an example), almost everything said so far generalizes immediately. Only 
the functional equations change. 


9.14 | The functional equations for shortest path problems from all nodes to all 
other nodes in a graph with no negative dicycles are 


vk, kl =Oallk == : = : 
UK, €] = min {ex (fk, i] + vf, i Ak, O} allk AL 


As with single-source cases, functional equations merely detail the prin- 


ciple of optimality ({9.10]) insight that optimal paths must have optimal subpaths. 
Thus the shortest path from k to £ must consist either of arc/edge (k, ¢), or an optimal 
path from k to some intermediate node i, plus an optimal path from i to 2. Sufficiency 
for computation follows exactly as it did for earlier model forms. 
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(a) Use inspection to identify optimal paths for all pairs. 

(b) Write the functional equations for (i, 7) = (1, 4) and (2, 3). 
(c) Verify that your optimal path lengths of part (a) satisfy these equations. 
Analysis: | 
(a) Optimal paths are detailed in the following table: 


j=l j=2 j=3 jz4 
bv, Path Vv Path Vv Path Vv Path 
1 (0) a 3 1-2 16 14-3 4 14 
2 3 2-1 0 —_ 19 2-1-4-3 7 2-1-4 
3 16 341 19 3-4-1-2 0 — 12 3-4 
4 4 1-4 dL, 4-1-2 12 4-3 0 — 


(b) Following , the functional equations for (i, 7) = (1, 4) and (2, 3) are 


v[1, 4] = min{c;,4, v[1, 2] + v[2, 4], v[1, 3] + v[3, 4]} 
v[2, 3] = min{cz3, v[2, 1] + v[1, 3], v[2, 4] + v[4, 3]} 


(c) Substitution of optimal values from the table in functional equations gives 


v[1, 4] = min{4,3+7, 16 +12} =4 
y[2, 3] = min{+00, 3 + 16,7 + 12} = 19 


Solving Shortest Path Problems by Linear Programming 


All the shortest path problems of this chapter can be formulated and solved > 
linear programming as long as the given graphs contain no negative dicycles. Deta™ 
are provided in Section 10.6. Still, it is important to understand that the dynam= 
programming methods of the remainder of this chapter are far more efficient. 


9.16 | Although shortest path 


problems over graphs with no negative dicycles 
can be solved by linear program 


3 SHORTEST PATHS FROM ONE NODE TO ALL OTHERS: 
BELLMAN-FORD 


We have seen that the task of computing shortest path lengths from single sours 
s to all other nodes reduces to a search for quantities v[k] satisfying functions 


9.3 Shortest Paths from One Node to All Others: Bellman—Ford 


equations [9.12]. Principle | 9.13} guarantees such v[k] will provide shortest path 
lengths whenever the given graph contains no negative dicycles. 


Solving the Functional Equations 


The central issue in designing shortest path algorithms is how to compute functional 
equation solutions efficiently. Notice that the task is not as straightforward as, say, 
solving a system of linear equations. The min{...} operator of our functional equa- 
tions makes them nonlinear. 

The format of equations suggests another way. Each equation shows an 
expression for a single value v[k]. Why not just evaluate those expressions? 

Sometimes we can employ such a one-pass evaluation strategy (see Sections 
9.6 and 9.7). But if the graph contains a dicycle like the following one: 


there is a difficulty. Notice that this dicycle has positive total length; the issue is not 
a negative dicycle. The difficulty comes from the form of the three corresponding 
functional equations: 


v[1] = min{v[3] + 5, ...} 

v2] = min{v[1] + 2, ...} 

v[3] = min{v[2] + 4,...} 
Evaluating the expression for v[2] requires v[1]; evaluating the expression for v[3] 
requires v[2]; and evaluating the expression for v[1] requires v[3]. 


Algorithm 9A, which is attributed to R .E. Bellman and L. R. Ford, Jr., draws on the 
notion of evaluation of functional equations to produce an algorithm that does work 
for any graph with no negative dicycles. The key insight is repeated evaluation. Each 
major iteration of the search (i.e., each f) evaluates the functional equation for each 
»[k] using results from the preceding iteration. We stop when no result changes. 

Just as with all the other searches of this book, Algorithm 9A distinguishes 
search values at different iterations by attaching superscripts. That is, 


v®[k] A value of v[k] obtained on the rth iteration 


When the algorithm finishes, we may want to know both shortest path lengths 
»[k] and the actual paths that achieve those optimal values. Labels d [k] keep notes 
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Step 0: Initialization. With s the source node, initialize optimal path lengths 


0 ifk=s 
(0) 
ied +oo otherwise 


and set iteration counter t < 1. 
Step 1: Evaluation. For each k evaluate 


vk] — min{v"P [i] + cix : Gk) exists} 


If L[A] < v“Y[k], also set d[k] <— the number of a neighboring node i 
achieving the minimum v®[k]. 

Step 2: Stopping. Terminate if v[k] = v’—[k] for every k, or if t = the 
number of nodes in the graph. Values v[] then equal the required shortest 
path lengths unless some v[k] changed at the last f, in which case the graph 
contains a negative dicycle. 

Step 3: Advance. If some v[k] changed and t< the number of nodes, 
increment ¢ < t+ 1 and return to Step 1. 


that we will see allow us to recover the optimal paths. Specifically, 


d[k] & node preceding k in the best known path from s to k 


Bellman-Ford Solution of the Two Ring Circus Example 


Before dealing with all the details of Bellman-Ford Algorithm 9A, let us apply it to 
the Two Ring Circus example [Figure 9.4(c)]. Table 9.1 provides details. 


TABLE 9.1 Bellman—Ford Algorithm Solution of Two Ring Example 


a eRe A Ee aA ee SS DTP 2p Pi 
tv) v2) v3] veal ves] ve) vr vrs) vr] 
0 0 +00 +00 +00 +00 +00 +00 +00 +00 
1 3.6 2.8 4.6 

2 3.7 41 

3 Sa 3.2 

4 4.8 6.7 
5 5.5 
6 
a en a Oe GR ee ee a 
t| d{i]  d[2]  d[3] = d[4]_—s d[5]_—s d{6]_—ss [7] [8] — a9] 
il 1 1 1 

2 2 3 

3 5 5 

4 G7; 7 
5 8 
6 


The process begins by initializing node values v[k]. Lacking better information, 
the algorithm initializes all except v[s] at the worst possible value for a minimizing 
optimization, -+oo. 


9.3. Shortest Paths from One Node to All Others: Bellman—Ford 


The first main iteration of the algorithm computes new v [k] by evaluating the 
functional equations using the v©[k]. For example, 


yO] = min{v[1] Se Cis vOU4] + C42} 
= min{0 + 3.6, co + 0.2} 
= 3.6 


Values change for v? [2] = 3.6, v [3] = 2.8, and v [4] = 4.6. 

Whenever a new value is assigned to any path length v[k], we also want to 
record the decision option that produced it. Here that means keeping track of the 
neighboring node i through which a new value for v[k] was derived. For example, 
4{2] <— 1 because the minimum establishing v [2] was achieved by extending the 
dest known path to neighboring node i = 1. Similarly, d[3] <— 1 and d[4] <1. 

Advancing to iteration t = 2, we repeat the process. This time, values change 
atnodes k = 4 and 5. The first of these illustrates the temporary nature of v[k] values 
in the Bellman-Ford algorithm. At iteration 1 we set v = 4.6 because 


vO [4] = min{v [1] + c1,4, v[2] + c2,4, PLB] + 3,4, v [5] + 5,4, V [6] + 6,4} 
= min{0 + 4.6, 00 + 0.1, 00 + 2.3, 00 + 1.5, 00 + 2.5} 
= 4.6 
After iteration 2, however, updated values produce 


v4] = min{v®[1] + 1,4, v [2] + c2,4, v3] + c3,4, v [5] + 5,4, v [6] + 6,4} 
= min{0 + 4.6, 3.6 + 0.1, 2.8 + 2.3, 00 + 1.5, 00 + 2.5} 
iil 
Processing continues in this way through iterations t = 3,4,5, with better 
and better values for the v[k] being computed. However, evaluation of functional 


equations at t = 6 produces no changed values. This is the signal to stop and accept 
values v© [k] as optimal. Figure 9.8 shows corresponding optimal paths. 


Davenport 


FIGURE 9.8 Optimal Paths in Two Ring Example 
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peri pelneane Ford Aleoa an 9A to compute the lengths of shortest paths from 
source node s = 1 to all other nodes of the following graph. 


Analysis: Initialization Step 0 sets 
vO] =0 v[2] = v3] = v [4] = co 
On iteration f = 1, evaluation of functional equations gives 
vOl1] = 0 
vO[2] = min{v [1] + c12, v[4] + car} 
min{0 + 5, co + 2} = 5 (making d[2] = 1) 
vO[3] = min{v [1] + e1,3, v© [2] + c2,3, v [4] + c4,3} 
{ 
{ 


= min{0 + 8, co — 10, co + 3} = 8 (making d[3] = 1) 
vO[4] 


min{} 
HCO 


Continuing in tabular form, we have 


t{ vO) v1 vB) vr 
0 0 +00 +00 +00 
1 5 8 

2 -5 

3 

t| d{ij a{2] d[3] a{4] 
1 il 1 

2 2 

Se ee 

Iteration t = 2 revises v™[3] = —5, which makes d[3] = 2. The algorithm 


terminates when all values repeat at iteration ¢ = 3. 

Extracting final values from the table, the shortest path to node 2 has length 
v[2] = 5, and the shortest path to node 3 has length v[3] = —S. Since the final 
v[4] = oo, there is no path to node 4. 


9.3 Shortest Paths from One Node to All Others: Bellman—Ford 


Justification of the Bellman-Ford Algorithm 


Having seen how Algorithm 9A works, we now need to investigate why. What 
guarantees that the v[k] correspond to optimal values when termination criteria 
are fulfilled? 

Look again at the complex of optimal paths in Figure 9.8. Some nodes are one 
arc away from the source along optimal paths, some are two arcs away, some three, 
and so on. Notice, however, that none is more than 


number of nodes —1=8 


arcs away. A path in a 9 node network can contain no more than (9 — 1) = 8 arcs. 
With any greater number, some node would have to repeat. 

The value v[1] = 0 at the source is correct from ¢ = 0. After iteration ¢ = 1, 
all optimal lengths that directly depend upon it (i.e., all v [k] with optimal paths of 
1 arc) must also be correct. Similarly, after t = 2, those v[k] with optimal paths of 
2 arcs must also be final. In general, intermediate v[k] reflect optimal paths of ¢ or 
fewer steps. All must be final before t = the number of nodes. Furthermore, if we 
go a whole iteration without changing any node’s value, no more changes will occur. 
We might as well terminate computation. 


Recovering Optimal Paths 


Some shortest path models require only shortest path lengths, but many require the 
optimal paths that achieve those lengths. 

‘To recover optimal paths, we exploit a consequence of principle of optimality 
9.10]. When optimal paths must have optimal subpaths, the shortest path to any 
node is just a 1-arc/edge extension of the shortest path to a neighbor. This implies a 
record of that last link is enough to recover the full path. We need only move to the 
indicated neighbor and follow its optimal path. 

The d[k] labels we record during computation provide the needed information 
because they record the last link of the sequence establishing optimal path length 
v[k]. 


To illustrate, consider the Lincoln-to-Tallahassee path we really need in Two 
Ring Circus Figure 9.8. Starting at Tallahassee node 9, final d[9] = 8 tells us the 
optimal path enters through node 8. Since the optimal path to node 9 must include 
an optimal subpath to node 8, we can continue the process by focusing on the shortest 
path to node 8, There d[8] = 7 tells us the path comes from node 7. 

Continuing through d[7] = 5, d[5] = 3, and d[3] = 1 we finally reach the 
source. Thus the full optimal path is 1-3-5-7-8-9: Lincoln to Wichita to Little Rock 
to Jackson to Montgomery to Tallahassee. 
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9.10: RECOVERING PATHS WITH BELLMAN-FORD 


een to the graph of Sample Exercise 9.9. Recover an optimal path to node 3 using 
the d[k] labels assigned during Sample Exercise 9.9 processing. 


Analysis: Applying principle [9.18 , we begin at destination node 3. Since d[3] = 2, 
the optimal path enters from node 2. The optimal path to node 2, in turn, enters 


through node d[2] = 1. Having reached the source, we conclude that a shortest path 
is 1-2-3. 


Encountering Negative Dicycles with Bellman—Ford 


We know that shortest path models with negative dicycles usually cannot be solved 
by the dynamic programming methods of Algorithm 9A (principle ). But if we 
confront a very large model, it could be quite difficult to know whether negative 
dicycles are present. 

To see what happens, we could try applying the Bellman—Ford algorithm to our 
negative dicycle example of Figure 9.6: 


vO} v2] vO] vl) | dt] [2] [3] [4] 


Initial values v[k] are chosen just as in other examples, and each iteration 
updates v[A] and d[k] labels. The critical difference is that labels continued to change 
on iteration tf = 4, even though the graph has only 4 nodes. Evaluation around 
negative dicycle drives v lower and lower because 

vO[2] <— v-Y[4] — 25 
y[3] — v@-)[2] +3 
vO[4] — v3] + 12 


This behavior illustrates why the Bellman—Ford algorithm terminates if labels 
continue to change on iteration t = the number of nodes. 
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We can actually exhibit a negative dicycle by following the d[k] labels of any k for 
which v[k] changed on the final iteration. For example, we could begin at k = 2 
because v [2] = —8 4 v2] = —3. Tracing backward through the a[k] labels, we 


find that 
a\2)| =4 
a[4] = 3 
a3 =2 


As soon as node 2 repeats, we know we have completed negative dicycle 2-342. 
9,4. SHORTEST PATHS FROM ALL NODES TO ALL OTHERS: 
FLOYD-WARSHALL 


When shortest paths between all pairs of nodes are required in a graph with no 
negative dicycles (e.g., in our Littleville example of Figure 9.1), the task is to compute 


v[k, €] efficiently, satisfying all-to-all functional equations ; 


Floyd-Warshall Algorithm 


Algorithm 9B, which is attributed to R. W. Floyd and S. Warshall, does just that. 
Search quantities 


v[k, €] & length of a shortest path for k to £ using only 
intermediate nodes numbered less than or equal to ¢ 


converge to the required shortest path lengths v[k, £]. Corresponding decision labels 
d[k, ¢] 5 node just before ¢ on the current path from k to £ 


track the associated paths. 

The key to Algorithm 9B’s effectiveness is the clever sequence in which quan- 
tities are calculated. Initialization correctly sets the v [k, £] = cxe because the only 
path from k to ¢ that has no intermediate nodes (i.e., none with positive node number 
< 0) is an arc/edge (k, 2) with cost Ck: 

Subsequent iterations consider concatenating previous results as indicated in 
the following sketch: 


By iteration t we already know optimal paths using nodes numbered < ¢ as inter- 
mediaries. Thus v[k, 2], which is the shortest using nodes numbered < t, must be 
either the current best or one visiting t via an optimal path from & to t followed by 
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Step 0: Initialization. All nodes should have consecutive positive numbers 
starting with 1. For all arcs and edges (k, 2) in the graph, initialize 


vOUk, £] — cee 
d[k,f] <k 
For k, £ pairs with no arc/edge (k, 2), assign 


0 ifk—2£ 
+oo otherwise 


vOrk, £] < 


Also set iteration counter t <— 1. 
Step 1: Evaluation. For all k, 2 4 ¢ update 


v[k, €] <— min{v Ik, €], v-?[k, tf] + ve, e]} 


If v[k, £] < vk, £], also set d[k, £] < d[t, 2]. 

Step 2: Stopping. Terminate if t = the number of nodes in the graph, or 
it vO[k,k] < 0 for any node k. Values v[k, €] then equal the required 
shortest path lengths unless some v[k, k]:is negative, in which case the 
graph contains a negative dicycle through k. 

Step 3: Advance. If ¢ < the number of nodes and all v[k, k] = 0, increment 
t <-t+1 and return to Step 1. 


one from f to £. The latter subpaths must involve intermediaries numbered < t, so 
they are already recorded in labels v“~ [k, t] and v“~P[z, £]. At termination, all the 


possibilities of the minimization in functional equations have been checked, 
which implies that the final v[k, ¢] are optimal. 


Floyd-Warshall Solution of the Littleville Example 


To illustrate Floyd—Warshall Algorithm 9B, we apply it to the Littleville example of 
Figure 9.1. Table 9.2 presents initial values, along with the results for iterations ¢ = 1, 
9, and 10. 

The algorithm begins by initializing v [k, ¢] for all arcs/edges (k, £) at direct 
cost cx,e, and fixing labels d[k, £] accordingly. For example, v[6, 7] <— c6,7 = 28, 
and d[6,7] < 6 to indicate that the current (direct) path from node 6 to node 7 
enters 7 via 6. 

Values for which there is no arc/edge (k, 2) begin at too except for k-to-k 
path lengths v[k, k], which are assigned value 0. Thus v[1, 10] < oo in Table 
9.2, because the Littleville network has no direct link from node 1 to node 10, and 
v[9, 9] — 0 to indicate that the shortest path from node 9 to itself has length = 0. 

Iteration t = 1 advances to Algorithm 9B, Step 1. For all node pairs k and £, 
both different than t = 1, we update v[k, £] to the minimum of what it was before 
and the concatenation the current path from k to t with the current path from £ to £. 

Table 9.2 shows that the only new (boxed) value occurs at k = 5, £ = 2. There 
is no direct path from node 5 to node 2 [refer back to Figure Example 9.1(b)], so 
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TABLE 9.2. Floyd—Warshall Algorithm Solution of Littleville Example 


Tnitial Values 

vVOOK@] €=1 €=2 €=3 f€=4 f=5 €=6 C=7 C=8 £=9 £=10 
k=1 0 12 [e-e) lore) [oe] lee) love) lo) [ere] co 
Ke?) lore) 0 18 co lore) 32 lore) oo fee) foe) 
ke 3 lee) foe) 0 13 lee) lore) 30 lore) lore) lo) 
k=4 fee) lee) lee) 0 [oe] co co co lee) 38 
kes 20 le) lo) [oe] 0 18 love) lo) oo co 
k= 6 oo Bp lo) co 18 0 28 foe) 25 lore) 
Kea lee) lee) 30 lo) love) 28 0 fore) 21 49 
k=8 foe) oo lee) [ee] 18 co foe) 0 36 oo 
k=9 lee) foe) lee) lee) foe) D5 21 36 0 40 
k=10 oo co foe) lero) foe) lo) 49 28 40 0 
d{[k, £] CO ol 2 ALT NDT WET CY one ES Sus 8 7 Tt Jom Get ee BE) 
Ke—olt — 1 — = 

k=2 — — 2 — — 2, 

e=3 3 — — 3 —_— — — 
k=4 — 4 
ESS) 5 — — — — 5 

k=6 — 6 — 6 — 6 — 6 —_ 
kav — — 7 — — 7 — — ul iD 
k=8 — — — _ 8 — _ — 8 — 
k=9 — — — — — 9 9 — 9 
k=10 — — — — — — 10 10 10 — 
After Iteration t= 1 

vO[kjé] €=1 ¢€=2 ¢=3 €=4 £=5 C=6 C=7 L=8 £=9 £=10 
Keel 0 12 lo) lee) lo) foe) oo foe) co oo 
k=2 fore) 0 18 ioe) lee) 32 foe) fore) fore) fore) 
ke3 co co 0 13 [ee] fore) 30 lo) lee) foe) 
k=4 co co lore) 0 lore) fee) co foe) co 38 
k= 5 20 co co 0 ‘18 co oo co co 
k=6 fore) 32 foe) co 18 0 28 lore) 25 co 
[c= co co 30 co fore) 28 0 co 21 49 
k=8 co [oe] lee) lee) 18 foe) co 0 36 foe) 
k=9 fore) lee) fore) foe) fore) 25 21 36 0 40 
k=10 fore) fore) co oo foe) lee) 49 28 40 0 
a{k,£] €=1 ¢€22 €=3 €=4 €=5 f€=6 ¢C=7 f=8 l=9 £=10 
Kao — 1 

k= 2 — — 2 —_— — 2 

b= 3 3 — — 3 — — — 
k=4 4 
k= 5 9) ] — — — 3) 

k=6 — 6 — — 6 —_— 6 6 — 
K=7 — —_ fh — — 7 —_ — 7 7 
k=8 8 — —_ — 8 — 
k=9 9 9 9 a 9 
k=10 — —_— — — — — 10 10 10 — 


y[5, 2] = +oo. But iteration t = 1 evaluates node 1 as an intermediary. The length 
of the path 5-1-2 is recorded-by evaluating the corresponding functional equation 
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TABLE 9.2 Floyd—Warshall Algorithm Solution of Littleville Example 


(Continued) 


After Iteration t= 9 


vO IK] €=1 €=2 €=3 ¢=4 f=5 C=6 C=7 C=8 f=9 £=10 
k=1 0 12 30 43 62 44 60 ~—* [105 69 81 
=2 70 0 18 31 50 32 48 93 57 69 
k=3 96 90 0 13 76 58 30 87 51 51 
k=4 lo) lo) loo) 0 oo fore) oo lo) lore) 38 
k=5 20 32 50 63 0 18 46 79 43 83 
k=6 38 32 50 63 18 0 28 61 25 65 
k=7 66 60 30 43 46 28 0 is7|| 22 21 49 
k=8 38 50 68 81 18 36 [57] 0 36 [76] 
k=9 63 57 51 64 43 25 21 36 0 40 
k=10 66 78 79 92 46 64 49 28 40 0 
d{k,f] €=1 €=2 £=3 ¢=4 &f=5 €=6 f=7 £=8 C=9 L=10 
k=1 aE 1 2 3 6 2 3 9 6 4 
k=2 5 a 2 3 6 2 3 9 6 4 
k=3 5 6 = 3 6 7 3 9 7 4 
k=4 4 
k=5 5 1 2 3 = 5 6 9 6 9 
k=6 5 6 2 3 6 ae 6 9 6 9 
k=7 5 6 a 3 6 7 ae 9 ii vi 
k=8 5 1 2 3 8 5 [9] = 8 p] 
k=9 5 6 7 3 6 9 9 OT poe 9 
k=10 5 1 ii 3 8 5 10 10 10 = 
After Iteration t = 10 
VOOTKL] €=1 C€=2 £=3 C=4 C=5 C=6 f=7 £=8 l=9 £=10 
k=1 0 12 30 43 62 44 60 105 69 81 
k=2 70 0 18 31 50 32 48 93 57 69 
k=3 96 90 0 13 16 5 30 51 51 
k=4 104] [116] [117 0 [84] [102] ~—‘ [87] 66 38 
k=5 20 32 50 63 0 18 46 79 43 83 
k=6 38 32 5063 18 0 28 61 25 65 
k=7 66 60 30.» «43 46 28 0 57 21 49 
k=8 38 50 68 = 81 18 36 57 0 36 76 
k=9 63 57 51 64 43 25 21 36 0 40 
k=10 66 78 19 ed 46 64 49 28 40 0 
d{k,f) €=1 ¢=2 ¢=3 ¢=4 ¢=5 ¢=6- ¢=7 f=8 b=9 £=10 
k=1 ae 1 2 3 6 2 3 9 6 4 
k=2 5 = 2; 3 6 2 3 9 6 4 
k=3 5 6 = 3 6 7 3 7 4 
k=4 5] — 3] 5] [fo] [10 fio) 4 
k=5 5 1 2 3 a 5 6 9 6 9 
k=6 5 6 D 3 6 ss 6 9 6 9 
k=7 5 6 7 3 6 eos 9 7 7 
k=8 5 il 2 3 8 5 9 i: 8 9 
k=9 5 6 7 3 6 9 9 9 = 9 
k=10 5 1 7 3 8 5 10 10 10 = 


vO[5, 2] <— min{v[5, 2], v[5, 1] + vf, 2} 
= min{oo, 20+ 12} 
—, 
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Decision label d[5, 2] is also corrected to indicate that the current path from k = 5 
to € = 2 now enters through node d[5, 2] = 1. 
Skipping ahead, Table 9.2 shows other values changing as nodes t = 9 and 
t= 10 are tried as intermediate nodes. One is 
v.13, 8] <— min{v[3, 8], v[3, 10] + v[10, 8]} 

= min{87, 51 + 28} 

=a 
Another value changed at ¢ = 10 is 

vOOl4, 1] <— v4, 10] + v[10, 1] 
= 38+ 66 = 104 


Look carefully at the corresponding d[4, 1]. Notice that it is set to d[10, 1] = 5, not 
?= 10. We want d/4, 1] to be the next-to-last node of the newly recorded path, which 
is information available in d[10, 1]. This may or may not equal the intermediate f. 

After iteration t = 10 every node has been tried as an intermediary; the 
Littleville network has only 10 nodes. Thus the algorithm terminates, and values 
v"[k, £] are optimal. That is, they provide the lengths of shortest paths from each 
k to each £. 


| SAMPLE EXERCISE 9.11: APPLYING THE FLOYD-WARSHALL 


Consider the following digraph: 


Apply Floyd-Warshall Algorithm 9B to compute the lengths of shortest paths be- 
tween all pairs of nodes. 


Analysis: Paralleling Table 9.2, we show tables of v [k, £] and d[k, £], boxing changed 
values. 


vOrk, £] 
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VOLK, 2] =v[k, 2] dk, £] 
k @€=1 ¢€=2 €=3 ¢€=4!] €=1 fC=2 ¢€=3 £24 
1 0 [e-e) (o-e) a — —_ _ 1 
2 20 0 00 2 2 a) = 2: 
Sita or) 0 3 == — i 
4 7 oo 4 0 4 pe 4 =e 
vk, £] d[k,£] 
k f=1 €=2 €=3 €=4| €=1 ¢€=2 &£=3 bC=4 
i 0 foe) [e-e) 7 — — — 1 
De DH 0 or) 2) 2 = = 2 
gy 3 oo 0 4 3 = = 1 
4 1 oo 4 0 3 — 4 = 
vOlk, 2] d[k,] 
k @=1 €=2 €=3 €=4] €=1 ¢€=2 C=3 b=4 
1 0 ror) fi] 7 = = 4 1 
D 3 0 6 2 3] = 4 2 
3-183 0° 0 4 3 = = 1 
4 1 ova) 4 0 3 = 4 = 


The table for t = 1 is identical to mat of f = 2 because no paths use node 2 as an 
intermediary. 

With only 4 nodes in the graph, computations are complete after iteration t = 4. 
Value v[1, 2] = 00 indicates that there is no path from node 1 to node 2. 


Recovering Optimal Paths 


Just as with earlier cases (principle ), final labels d[k, £] allow us to recover an 
optimal path between any pair of nodes. 


aa teelf i is ‘encountered. 


To illustrate, refer again to final tf = 10 labels for the Littleville example dis- 
played in Table 9.2. A shortest path from node k = 3 to node ¢ = 8 is recovered by 
beginning at destination €. Label d[3, 8] = 10 indicates that an optimal path enters 
from node 10. Now backtracking to node 10, we apply principle of optimality 
and follow the best path from 3 to 10. Label d[3, 10] = 4 tells us that that path enters 
through node 4. Backtracking again, we discover label d[3, 4] = 3. Origin k = 3 has 
been reached, and the optimal path is 3-4-10-8. 


Use final labels of Sample Exercise 9.11 to recover an optimal ay from k = 2 to 
CaN 
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Analysis: Applying principle [9.20], we backtrack from destination £ = 1. Label 
#(2, 1] = 3 indicates the optimal path enters through node 3. Continuing, the optimal 
path from 2 to 3 enters through d[2, 3] = 4, and the one for 4 enters through d [2,4] = 
2. which brings us to origin k = 2. Thus the optimal path is 2-4-3-1. 


Detecting Negative Dicycles with Floyd-Warshall 


Just as with the Bellman—Ford algorithm for a single source, Floyd—Warshall Algo- 
rithm 9B guarantees optimal paths and path lengths only if the given graph contains 


no negative dicycles. Functional equations | 9.14], on which the algorithm is based, 
need not hold in the presence of negative dicycles. 

What happens when Algorithm 9B is applied to a graph with negative dicy- 
cles? The secondary stopping criterion of algorithm Step 2 comes into play, and 
computation terminates with the conclusion that a negative dicycle exists. The actual 
condition flagging presence of a negative dicycle in the Floyd—Warshall procedure is 
v Tk, k] < 0 (i.e., a shortest path from any k to itself turning negative). 


21 _| If Algorithm 9B is applied to a graph with negative dicycles, it will demon- 
strate their presence by making some v[k, k] < 0 and terminating. Under 


To illustrate, return to the negative dicycle example of Figure 9.6. Application 
of Algorithm 9B produces the following: 


VOrK, 2] =v Lk, £] d[{k,£] 
ea a2) a3) Saale vane ea Sa 
1 0 2 10 [oe) — 1 1 —_— 
2 lore) 0 3 fore) — — 2, — 
ah fore) love) 0 12 — — — 3 
4 lo-e) -25 lee) 0 — 4 — —_ 
vk, £] d[k, £] 


k = 3 = 
il 0 2 5 17 = 1 2 13] 
2 ee) 0 3) {15} — — 2 3] 
3 [e-e) ie.) 0 12 — — 3} 
4 ee) 25 22 4 3 13] 


Notice that on iteration ¢t = 3, value 

v[4, 4] — min{0, v©[4, 3] + v®[3, 4]} 
min{0, —22 + 12} 
—10 
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This would cause the algorithm to terminate at Step 2 with the conclusion that a 
negative dicycle involving node 4 is present in the graph. 

As usual, final d[k, ¢] labels allow us to retrieve the dicycle if needed. Starting 
at node 4 in the table for ¢ = 3, 


d[4, 4] =3 
d[4,3] =2 
d[4,2] = 4 


indicating the negative dicycle 4-2-3-4, 


9,5 SHORTEST PATH FROM ONE NODE TO ALL OTHERS 
“WITH Costs NONNEGATIVE: DIJKSTRA 


The Bellman—Ford and Floyd—Warshall algorithms of Sections 9.3 and 9.4 can solve 
any of the shortest path models presented in Section 9.1 and any of the others we will 
encounter in Sections 9.6 and 9.7. They require only that the given graph contains no 
negative dicycle. Still, those two algorithms are not the most efficient option when 
given graphs satisfy further assumptions. In this section we develop the algorithm 
credited to E. W. Dijkstra, which is considerably more efficient for cases where we 
need shortest paths from one node to all others, and all costs are nonnegative (c;; > 
0). Algorithm 9C provides a formal statement. 


ALGORITHM 9C; ONE TOALL (NONNEGATIVE COSTS); DUKSTRA 


Step 0: Initialization. With s the source node, initialize optimal path lengths 
sale 0 Ls 
+oo otherwise 


Then mark all nodes temporary, and choose p < s as the next permanently 
labeled node. 

Step 1: Processing. Mark node p permanent, and for every arc/edge (p, i) 
leading from p to a temporary node, update 


v[i] — min{v[i], v[p] + cp,i} 
If v[i] changed in value, also set d[i] < p. 
Step 2: Stopping. If no temporary nodes remain, stop; values v[i] now 
reflect the required shortest path lengths. 


Step 3: Next Permanent. Choose as next permanently labeled node p a 
temporary node with least current value v{i], that is, 


v[p] = min{v[i] : i temporary} 


Then return to Step 1. 


Permanently and Temporarily Labeled Nodes 


A graph with nonnegative arc/edge costs certainly contains no negative total length 


dicycles. Thus functional equations apply. 


9.5 Shortest Path from One Node to All Others with Costs Nonnegative: Dijkstra 


What is new about Dijkstra Algorithm 9C is how values satisfying functional 
equations are computed. Bellman—Ford Algorithm 9A simply evaluates functional 
equations for all nodes on all iterations. Thus it processes all the inbound arcs and 
edges at a node many times while computing the minimum in [9.12 

Dijkstra’s method processes outbound arcs and edges instead of inbounds. 
Much more important, it processes each arc/edge only once. Each major iteration 
makes one new node p permanent. 


9.22 | Once a node is classified permanent by Dijkstra Algorithm 9C, its a 
and d[p] labels never change again. Nodes a are not yet permanently labeled 
are classtied ae 


As each new permanent node p is selected, we correct 
v[i] — min{v[Z], v[p] + cp} 
at all temporarily labeled neighbors i reachable from p. Knowing that the labels for 
p are final, we need never again consider arc/edge (p, i) in our search for the final 


value of v[i]. This finalizing of a node label at each step is what makes Algorithm 9C 
the best for its class. 


Dijkstra Algorithm OC is the most efficient method available for compute 


faving all arcledge costs nonnega ive 


Of course, even faster schemes can be developed if the given graph or digraph has 
special features beyond nonnegative costs (see Section 9.6). 


Least Temporary Criterion for Next Permanent Node 


It should be obvious that the heart of Dijkstra’s algorithm is a rule for selecting the 
next temporary node to make permanent. The one required is elegantly simply: 


5 24 | Bach oes 3 Distr MENA dese eats 


All current v[i] on temporary nodes are examined, and a p is selected with 


v[p] = min{r[Z] : i temporary} 


Dijkstra Algorithm Solution of the Texas Ti pasler ee a 


Before investigating why criterion |' 
9C to the Texas Transfer example reproieed in Figure 9.9. Table 9.3 provides 
computational details. 

Solution begins in exactly the same way as previous methods. Values v[i] <— co 
except at the source node 3 with v[3] < 0. All nodes start temporary. Source 3 is 
automatically the first permanent. 
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El Paso 


Corpus Christi 


FIGURE 9.9 Texas Transfer Example Network 


Processing a new permanent node p means checking the v[i] values of all tem- 
porary nodes i reachable from p in one step. For p = 3 these include nodes 1, 5, 8, 


and 9. Thus 
vy[1] <— min{v[1], v[3] + 3,1} 
= min{oo, 0 +359} 
= 1359 
v[5] <— min{v[5], v[3] + ¢3,5} 
= min{oo, 0 +180} 
= 180 
v[8] — min{v[8], v[3] + c3,8} 
= min{oo, 0+195} 
= i195 
v[9] <— min{v[9], v[3] + ¢3,9} 
= min{oo, 0 + 246} 
= 246 
Corresponding labels d[1] <— d[5] < d[8] <— d[9] < 3 because all four v[i] changed 
from 3. 


It is now time to apply criterion . The next permanent node p must be a 
temporary i of minimum v[/]. With all nodes temporary except 3, 


9.5 Shortest Path from One Node to All Others with Costs Nonnegative: Dijkstra 


TABLE 9.3 Dijkstra Algorithm Solution of Texas Transfer Example 


H-| 


v[qJ v[2] v[3] v[4] v[5] v[6] v[7] v[8] v[9] v[10] 


ee a el 
mit) fore) foe) 0 ee) (ore) [ee) (oe) (oa) ore) (oa) 
EEE EE eS a ae ae ee eee 
3 359 (perm) 180 195 246 
5) 347 623 (perm) 242, 
8 274 (perm) 
9 (perm) 461 
6 (perm) 
7 (perm) 427 
1 (perm) 
2 (perm) 
10 (perm) 
4 (perm) 
—_— EE EEE ee ee ee ee a ee ee Se ee 
final) 359 347 0 623 180 272 274 195 246 427 
Dp d{i] d[2] a[3] a[4] a[5] d[6] a{7] d{8] a[9] d{10] 
3 5 3 3 3 
5 5 5 5S) 
8 8 
) 9 
6 
7 7 
1 
2 
10 
4 


—_CoO...kl( QGeaQeGe—— es 


final) 


3 5) _ 5 3 =) 8 3 3 7 


min{y[1], v[2], v[4], v[5], v[6], v[7], v[8], v[9], v[10]} 
= min{359, oo, ov, 180, 00, 00, 195, 246, oo} 
= 180 


The smallest value occurs at node 5 and p = 5. 


Corresponding d[2] < d[4] < 


Correcting labels on outbound edges gives 


v2] — min(v2], v[5] + es) 
= min{oo, 180 + 167} 
= 347 

y[4] <— min{v[4], v[5] + cs,4} 
= min{oo, 180 + 443} 
= 623 

v[6] <— min{v[6], v[5] + cs6} 

min{oo, 180 + 92} 

= 272 


d[6] —5. 
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To choose the next p, we examine the remaining temporary nodes: 
min{v[1], v[2], v[4], v[6], v[7], v[8], v[9], v[10]} 
= min{359, 347, 623, 272, oo, 195, 246, oo} 
— hs) 
Here p = 8, leading to v[7] <— 195 + 79 = 274 and d[7] < 8. 
The next permanent node is p = 9 because 
min{v[1], v[2], v[4], »[6], v[7], vf9]. v[LO]}} 
min{359, 347, 623, a4 a 246, co} 
= 246 


Processing yields 


v[7] <— min{v[7], v[9] + co,7} 
= min{274, 246 + 199} 
S24 

< min{v[10], v[9] + c9.10} 
= min{oo, 246 + 215} 

= 461 


Notice that v[7] did not change. Thus only label d[10] < 9. 

Remaining processing of Table 9.3 follows ina similar way. Successive iterations 
make permanent nodes 6, 7, 1, 2, 10, and 4. When all nodes are permanent, the 
algorithm terminates. Final v[i] now represent the required lengths of shortest paths 
from source node 3. 


v[10 


= 


SAMPLE EXERCISE 9.13: APPLYING DUKSTRA’S ALGORITHM 


Apply Dijkstra Algorithm 9C to compute the length of a shortest path con nda 
s = 1 in the following graph to every other node. 


Analysis: Paralleling Table 9.3, we may summarize computations in tables for v[z] 
and d[i]: 


Pp v[i] v[2] v[3] v[4] Vv[5] 
(init) 0 fore) co co fora) 

1 (perm) 5 20 5 

2 (perm) 17 

5 a (perm) 

3 (perm) 

4 (perm) 
(final) 0 5) W fore) 5 
a A ee Se ee ee See ee 
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P da{1] d[2] d[3] ad[4] a{5] 
1 1 1 1 
2 2 

5 5 

3 

4 

Gna fine ae 1 


One new feature of this example occurs with the selection of the second permanent 


node p = 2. Criterion requires us to choose a temporary node i of minimum 
v[i], but there is a tie. Both v[2] = 5 and v[5] = 5. Either could be selected. The 
computations above arbitrarily chose p = 2. 

One other different element is the presence of a node i = 4 to which there is 
no path. As with other shortest path algorithms, this condition is indicated by final 
v[4] = oo. 


Recovering Paths 
The Texas Transfer example required only shortest path lengths, not the paths them- 
selves. However, decision labels d[i] make it possible to recover any needed paths 


using principle [9.18]. For example, the shortest path from source node s = 3-to 
i = 10 could be traced backward from i = 10 as 


d{10] =7 
A208 
d[8] =3 


The resulting shortest path is 3-8-7-10. 


Justification of the Dijkstra Algorithm 
The Dijkstra algorithm designation of permanently labeled nodes certainly speeds 


computation. But how do nonnegative costs and selection criterion make it 
work? 
The key insight is an interpretation of v[i] values during computation. 


nodes. ee — a 

That is, interim results reflect optimal paths through nodes already classified perma- 
nent. 

For an example, focus on v[10] in Table 9.3. There is no path from s = 3 to 
i = 10 in the Texas Transfer network of Figure 9.9 that uses only nodes 3, 5, and 8. 
That is why v[10] = oo through iterations making those nodes permanent. When 
node 9 becomes permanent on the fourth iteration, an all-permanent path to node 10 
does become available. Value v[10] = 461 to account for that path 3-9-10. Although 
the path is shortest among nodes classified permanent through 4 iterations, it is not 
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the shortest overall. Optimal path 3-87-10 (length v[10] = 427) is discovered only 
2 iterations later when node 7 is made permanent. 


With interpretation , we can easily see why the node p chosen by criterion 
9.24 | is ready to become permanent. 


first 
temporary 


Label v[p] corresponds to the shortest path to node p using only permanently labeled 
nodes. Noting that s becomes permanent on the very first iteration, any other path 
begins among permanent nodes, passes along a subpath to a first temporary i, and 
completes with an i to p subpath. The length of such a path is 


(s-to-i subpath) + (i-to-p subpath) > v[i] + 0 
because v[i] is the length of a shortest all-permanent path to i, and arc/edges costs of 


the i-to-p subpath are nonnegative. But since p was chosen (criterion ) as the 
temporary node with smallest v-value, 


v[i] + 0 = vfi] = v[p] 


Thus no path can be shorter than the one for v[p], and we may justly term it perma- 
nent. 


0.6 SHORTEST PATHS FROM ONE NODE TO ALL OTHERS 
IN ACYCLIC DIGRAPHS 


The key to Dijkstra Algorithm 9C’s efficiency was careful selection of permanently 
labeled nodes, so that arcs needed to be processed only once. Computation becomes 
even more efficient if a sequence for permanent-node processing can be determined 
before we begin. 


Acyclic Digraphs 

A predetermined sequence is possible if the model arises on an acyclic digraph, which 
is a directed graph with no dicycles. Figure 9.10 illustrates the definition. Part (a) 
shows a graph that is acyclic because it has only arcs (no edges) and has no directed 
cycles (dicycles). Graphs of parts (b) and (c) fail the definition. The first is only 
partly directed; for example, it contains edge (2, 3). The second is fully directed, but 
contains dicycles; one is 2-34-52. 


9.6 Shortest Paths from One Node to All Others in Acyclic Digraphs 


eo — 
@) 

oS —-@ 

(a) Acyclic 

S—_© 
Q) 

y) 


(b) Las edges 


@) (4) FIGURE 9.10 Definition 


(c) Has dicycles of an Acyclic Digraph 


We can determine whether small graphs such as those of Figure 9.10 are acyclic 
simply by inspection. For larger graphs we require an easy-to-check condition. 


9, 26. =< digraph (completely directed graph) is acyclic if and only iti 
can be numbered so that every arc @ / hasi bap 


Figure 9.10(a) is already numbered in this way. Each arc goes from a lower- 
numbered to a higher-numbered node. Notice how this guarantees that the digraph 
contains no dicycles. Since each arc takes us to a higher-numbered node, no path 
could ever close a cycle by repeating a node that we have already visited. 

Conversely, it is not hard to find a suitable numbering for a digraph that is 
acyclic. 


a2) Any acyclic digraph can be number in principle [9.26] = 7 
transiting the graph in depth-first fashion, numbering nodes in decreasing Se 


quence as soon as all their outbo id to nodes already visited. 


Here depth-first simply means that we pass to new nodes before backtracking to ones 
already visited. 

We can illustrate by computing the node numbers of Figure 9.10(a). To avoid 
confusion, suppose that the graph was given to us with alphabetic node labels as 
follows: 
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(@) first 


Beginning arbitrarily at node a, we pass first to node b. Since it has no outbound 
arcs, we assign it the highest possible number, b = 5, and backtrack to a. There, an 
outbound arc remains to node d, and it has an arc to e. Node e can now be numbered 
e = 4 because it leads only to already visited node b = 5. Backtracking to node d, we 
now assign it number d = 3. Similarly, node a is numbered a = 2. Having completed 
all arcs from our origin a = 2, we must start again at some still unnumbered node. 
Here the only choice is c, which immediately becomes node c = 1. 


Determine whether each of the following digraphs is acyclic. 
@ 2 @ (>) 
@ So © ) 
(a) (b) 


(a) This graph is not acyclic. For example, it contains dicycle a—b—c—a. 


(b) This graph is acyclic. To demonstrate that it is, we apply principle . Starting 
at node a, depth-first processing proceeds as follows: 


Lal 
= 


te Ss. 


We advance to b, then c, then d before being blocked. Numbering d = 5 and back- 
tracking, we can now go to e. It leads nowhere, so we number e = 4 and backtrack 
to c. With no further unexplored neighbors, c = 3, and further backtracking yields 


b =2 and a= 1. Node numbers now satisfy condition 9.26 |. 


Analysis: 


9.6 Shortest Paths from One Node to All Others in Acyclic Digraphs 


Shortest Path Algorithm for Acyclic Digraphs 
An acyclic digraph certainly has no negative dicycles, because it has no dicycles at 


all. Thus functional equations do apply. 
Still, the most important convenience of acyclic digraphs in shortest path com- 


putation arises from the numbering of principle |9.26|. If we evaluate functional 
equations for nodes in that acyclic number sequence, each v[k] is permanent as soon 
as it is computed. This is true because all terms in equations involve inbound 


arcs from lower-numbered nodes with previously fixed v[i]. Algorithm 9D provides 
details, and it should be no surprise that it is the most efficient possible. 


Step 0: Initialization. Number nodes so that each are (i,j) of the digraph 
has i <j. Then set source s optimal path length 


vis] — 0 


Step 1: Stopping. Terminate if all v[k] have now been fixed. Otherwise, let 
p be the lowest number of an unprocessed node. 


Step 2: Processing. If there are no inbound arcs at node p, set v[p] < +00. 
Otherwise, compute 


v[p] <— min{v[i] + jp : @ p) exists} 


and let d[p] < the number of a node i achieving the minimum. Then return 
to Step 1. 


from one ae to all Others ¢ in acyclic digraphs. 


Acyclic Shortest Path Example 


None of the models of Section 9.1 are posed on acyclic digraphs, so we will illustrate 
Algorithm 9D on the digraph of Figure 9.11. Table 9.4 provides results. 
Notice that the digraph in Figure 9.11 is already numbered as in principle 5 
and we will assume that paths are to begin at sources = 1. Initialization sets v[1] < 0. 
Node processing now takes place in node number sequence. At node p = 2, 
values are updated as 
v[2] <— min{v[1] + cy} 
= min{0+5} 
=) 


with d[2] < 1. Then at node p = 3, we compute 


v[3] <— min{r[1] + cy3, v[2] + co,3} 
= min{0+8,5—10} 
= —5 
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FIGURE 9.11 Acyclic Shortest Path Example 


TABLE 9.4 Solution to Acyclic 
Shortest Path Example 


[viel_diel | p | virl_ ate 
6 


J 
0 


with d[3] < 2. Computation now continues to p = 4, p = 5, and so on, until we 
complete at p = 9. If required, shortest paths can be retrieved from final d[k] by 


applying principle | 9.18}. 


9,7 CPM PROJECT SCHEDULING AND LONGEST PATHS 


Section 9.1 introduced some of the more common shortest path model forms. How- 
ever, many other models are closely related, even though they appear at first glance 
to have nothing to do with computing paths. 


Project Management 


One of the most commonly occurring such problems arises in the management of 
large work projects. For planning and control purposes, projects are usually subdi- 
vided into a collection of work activities that must all be completed to accomplish 
the project. Each activity has an estimated duration, 


a; & time required to accomplish activity k 


and a list of predecessor activities. 


The main issue is to compute an early start schedule for the project. That is, we want 
to know the earliest time that each activity can begin, subject to the requirement that 
all its predecessor activities have already completed. 


9.7 CPM Project Scheduling and Longest Paths 


EXAMPLE 9.4: WE BUILD CONSTRUCTION 


For a contrived but suggestive example, consider We Build Construction’s latest 
project. We Build is developing a 1-story medical office building on available land 
near a hospital. 

Table 9.5 details the project’s 9 work activities. For example, the table shows 
the estimated duration of the heating and air conditioning activity k = 7 is a7 = 13 
days, and that activity 7 cannot begin until predecessor activities 2 = rough plumbing 
and 4 = structural members have been completed. 


TABLE 9.5 We Build Construction Example Tasks 


Duration, Predecessor 


k Activity a), (days) Activities 
1 Foundation 15 _ 

2 Rough plumbing 5 _ 

3 Concrete slab 4 HA 

4 Structural members 3 3 

5 Roof a 4 

6 Rough electrical 10 = 

7 Heating and air conditioning 13 2,4 

8 Walls 18 4,6,7 

9 Interior finish 20 5,8 


To plan materials deliveries and arrange subcontractors to do the various ac- 
tivities, We Build needs a schedule. In particular, they want to know the earliest time 
after project start that each activity can begin. 


CPM Project Networks 


To address project scheduling problems with shortest path technology, we require 
a network or graph. The critical path method (CPM) forms such project networks 
from precedence relationships. 


os each activity, Ares of zero. ea connect = to all aetivities without pre- 
dece eSsOrs. s. Other ares of length a, connect each activity node k to all activities 


Figure 9.12 illustrates for the We Build example of Table 9.5. The construction 
begins by creating a node for each of the 9 activities, plus special start and finish 
nodes to represent the beginning and ending of project activity. Each precedence 
relationship generates an arc with length equal to the predecessor’s duration. For 
example, arc (8,9) represents the fact that activity 8 is a predecessor of activity 9. Its 
length is ag = 18 days. 

Special arcs from start and to finish complete the digraph. Zero-length arcs 
lead from start to each activity having no predecessors. In the We Build example 
those are 1 = foundation and 2 = rough plumbing. Arcs of length a, join activities 
that are the predecessors of no other activity to finish. Here only activity 9 = interior 
finish qualifies. 
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The following table lists the. activities foaned ofa ooleeal Me acer team in arranging 
a campaign rally. Construct the corresponding CPM project network. 


Duration, Predecessor 
Activity a;, (days) Activities 


Contact local party 2 
Find location 1 
Arrange date and time 1 
Notify news media 1 
Arrange sound system 3 
1 
1 
1 


— 


Coordinate police security 
Install speaking platform 
Decorate platform and site 


AADAKRWNPH] & 
Nis 

WwW 
NAW WWNrR 


Modeling: Following construction , we obtain the following CPM project net- 
work: 


9.7 CPM Project Scheduling and Longest Paths 


CPM Schedules and Longest Paths 


Focus on activity 3 in Figure 9.12. When is the earliest time it can begin? One 
limit comes from predecessor activity 1 with duration 15. Another derives from 
predecessor activity 2 with duration 5. Both must be completed before activity 3 can 
begin. Since neither has a predecessor, both can begin at time 0. Thus the earliest 
start time for activity 3 is day 


max{da), a2} = max{15,5} =15 


Notice that the constraint imposed by predecessor 1 corresponds to path start— 
1-3 in Figure 9.12, and that of predecessor 2 corresponds to path start-2-3. The 
longest such path defines the early start time for activity 3. This is true in general. 


The longest path to node k is the longest succession of activities that must complete 
before activity k can begin. 


Critical Paths 


Such longest paths are also called critical paths, from which the Critical Path Method 
derives its name, because delay in any activity along the critical path to node k will 
delay the start of activity k. The most important is the critical path from start to 
finish, which links activities that could delay completion of the entire project if they 
extend beyond their planned durations. Early start times and critical paths for all 
activities in the We Build project are shown in the following: 


Early 
k Activity Start Critical Path 
1 Foundation 0 start—1 
2 Rough plumbing 0 start—2 
3. Concrete slab 15 start-1-3 
4 Structural members 19 start-1-3-4 
5. Roof 22 start-1-3-4-5 
6 Rough electrical DD, start-1-3-4-6 
7 Heating and air conditioning 22 start-1-3-4-7 
8 Walls 35 start-1-3-4-7-8 
9 Interior finish 53 start-1-3-4-7-8-9 


In a similar way the least possible duration of a project corresponds to the 
longest start-to-finish path. 


or citical path from start to finish i in the corresponding project network. 


For We Build, this project critical path runs 
start-1-3-4_7-8-9-finish 


with a total length of 73. Thus the minimum time to complete the building detailed 
in Table 9.5 is 73 days. 
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SAMPLE EXERCISE 9. 16: IDE NTIFYING CRITICAL PATHS © 


Use inspection to determine the earliest start times and mica hah for all activities 
of the political advance project in Sample Exercise 9.15. Also determine the mini- 
mum project duration and the activities which, if delayed, will delay completion of 
the whole project. 


Analysis: Inspecting for the longest paths in the network of Sample Exercise 9.15, 
start times and critical paths are as follows: 


Early 
k Activity Start Critical Path 
1 Contact local party 0 start-1 
2 Find location 2 start-1-2 
3 Arrange date and time 35 start-1-2-3 
4 Notify news media 4} start-12-3-4 
5 Arrange sound system 5 start-1—-2-3-5 
6 Coordinate police security 45 start-1-2-3-6 
7 — Install speaking platform Ts start-1-2-3-5-7 
8 Decorate platform and site 9 start-1-2-3-5-7-8 


Minimum project duration will correspond to the longest start-to-finish path 
start-1—-2-3-5-7-8-finish 


of length 10 days. Thus the activities that must complete as planned if the project is 
not to be delayed are activities 1, 2, 3,5, 7, and 8. 


Computing an Early Start Schedule for the We Build Construction 
Example 


Algorithm 9E details a longest path scheme for computing early start times. Table 
9.6 shows the results from application of Algorithm 9E to compute an early start 
schedule for our We Build example. Notation now signifies the following: 


v[k] & length of the longest path from source start to node k 


d[k] & immediate predecessor of node k in a longest path 


TABLE 9.6 Early Start Schedule 
for the We Build Example 


tel ate |e) 
0 6 


start 
start 


vip] 4{p] 


uo 
nn 
woonR 


9.7 CPM Project Scheduling and Longest Paths 


Step 0: Initialization. Number activity nodes so that each precedence arc 
(i, 7) of the CPM project network has i < j. Then initialize the schedule time 
of the project start node as 


v[start] — 0 


Step 1: Stopping. Terminate if the early start time of the project finish node 
has been fixed. Otherwise, let p be the lowest number of an unprocessed 
node. 

Step 2: Processing. Compute the activity p early start schedule time 


v[p] — max{v[i] + a; : i is a predecessor of p} 


and let d[p] < the number of a node i achieving the maximum. Then return 
to Step 1. 


As before, computation begins by initializing source node time v[start] <— 0. 
Then values for other nodes are established in increasing number order. First, 
v[1] <— max{v[start] + dstarc} 
= max{0+0}=0 
with d[1] < start. Ina similar way, v[2] < 0 and d[2] < start. Then, at p = 3 
v[3] <— max{v[1] + a1, v[2] + ap} 
= max{0+15,0+5}=15 
making d[3] <1. The final results in Table 9.6 correspond exactly to the early start 
schedule detailed above. 
Critical paths can be retrieved using optimal d[k]. For example, the entire 
project’s critical path 
start-1-3-4-7-8-9_finish 


is recovered by backtracking (principle ) from finish as 


d[finish] =9 
d[9]=8 

d[8] =7 

d{7] = 

d[4] =3 

d[3] =1 

d[1] = start 


9E to compute an early start schedule and identify the critial path for the entire project 
finish. 
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Analysis: Following Algorithm 9E, v[p] and d[p] are set in increasing node number 
sequence. Results are summarized in the following table: 


8 
finish 


The critical path from start to finish is traced d[finish] = 8, d[8] = 7, d[7] = 5S, 
d[5] = 3, d[3] = 2, d[2] = 1 and d[1] = srart, yielding 


start-1-2-3-5—7-8-finish 


Late Start Schedules and Schedule Slack 


Computations of early start Algorithm 9E implicitly assume that all activities can 
start at time 0. Early start schedules are then derived by finding the longest path of 


predecessor activity durations and summing (principle ). 

Suppose now that we have a due date for the project, that is, a time by which 
all activity must be completed. For example, our We Build project might be required 
to be complete on day 80. We can perform longest path computations in reverse to 
compute a late start time for each activity showing the latest it can begin if the due 
date is to be met. 


__9.33 | The latest start time of any activity k ina project equals the due date less 
the length of a longest path ion k to the finish node i in the conesponcing CPM 
network. 


To find this late start schedule, we simply reverse the sequence of computations 
in Algorithm 9E. Table 9.7 illustrates for the We Build example with a due date of 80. 

Computation begins by fixing the late start schedule for the finish node at the 
due date. Then we advance to the highest-numbered activity, 9. Its late start time is 
the minimum of those for successor activities less its own duration, or 80 — 20 = 60. 
Continuing in this way, the late start time for activity 4 is computed: 


v[4] = min{r[5], v[6], v[7], v[8]} — a4 
= min{53, 32, 29/42} — 3 
==. 


and so on, until we complete the start node. 
With both early and late schedules in hand, we can also subtract to determine 
the schedule slack for each activity. 


9.34 Schedule slack is ne diffi en 


yeen the arliest and latest times that 


an activity can begin. 


9.7 CPM Project Scheduling and Longest Paths 


TABLE 9.7 Late Start Schedule 
for the We Build Example 


vip] dipl |p | 
80 _— 4 


v[p] d[p] 


finish 
9 


8 
7 
6 
5 


That is, schedule slack shows how much discretion we have in scheduling any single 
activity while meeting the overall project due date. Table 9.8 computes such slacks 
irom the We Build example activities in Tables 9.6 and 9.7. 


TABLE 9.8 Schedule Slack for the We Build 


Example 
Early Late 

k Activity Start Start Slack 
1‘ Foundation 0 7 W 
2 Rough plumbing 0 17 17 
3. Concrete slab 15 22 7 
4 Structural members 19 26 7 
5 Roof 22 53 BIL 
6 Rough electrical 22 32 10 
7 Heating and air conditioning 22 29 dl 
8 Walls 35 42 il 
9 Interior finish 53 60 7 


SAMPLE EXERCISE 9.18: CoMPUTING LATE START SCHEDULES AND SLACK 


Return to the political advance project of Sample Exercises 9.15 and 9.17. Compute 
the corresponding late start schedule for each activity and its schedule slack assuming 
that all work must be completed by day 10. 


Analysis: As above, late start times are set in decreasing activity number sequence. 
Then differences are computed versus early start times to determine slack. The result 
is summarized in the following table: 


oN DAMN PW YH PY 


Early Late 
Activity Start Start Slack 
Contact local party 0 0 0 
Find location 2 2, 0 
Arrange date and time 33 33 0 
Notify news media 3 9 45 
Arrange sound system 45 45 0 
Coordinate police security 45 y) 45 
Install speaking platform 3 75 0 


Decorate platform and site 9 9 0 
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Difficulty of General Longest Path Problems 


Longest path problems are the maximize analogs of the shortest path problems which 
were solved very efficiently by Algorithms 9A to 9D. All the functional equation 
and algorithmic technology of Sections 9.2 to 9.7 can be adapted to treat longest path 
cases simply by substituting max for min in update operations. 

Notice, however, that the difficult case of negative dicycles in shortest path 
problems (principle [9.9]) becomes the case of positive dicycles in longest path 
models—dicycles of positive total length. 

Elementary dynamic programming methods usually cannot deal with longest 
path problems on graphs with positive dicycles. The fact that every one of the graphs 
of Section 9.1 contains positive dicycles explains why longest path problems are 
generally considered much less tractable than shortest path. Positive dicycles are far 
more common than their negative relatives. 


Acyclic Character of Project Networks 


Positive dicycles cannot occur in acyclic digraphs, which have no dicycles at all. We 
can do CPM scheduling and the associated longest path computations because CPM 
networks fit this acyclic assumption. 


O38 | Every well- formed project network is acyclic. 


To see why, recall that start nodes in construction (530) have shee ‘oulboumt 
arcs, and finish nodes have only inbound arcs. Thus any dicycle would have to run 
through other nodes (i.e., those for activities). But arcs connecting activity nodes 
correspond to precedence relationships. Any dicycle i-j-k-i 


would mean that activity i must precede activity j, activity 7 must precede activity 
k, and activity k must precede activity i. This is physically impossible. A project 
network with a dicycle admits no feasible schedule (if durations a, > 0). 


9,8 DISCRETE DYNAMIC PROGRAMMING MODELS 


Elementary discrete dynamic programming encompasses more than shortest paths, 
including a variety of problems that can still be modeled as shortest or longest path 
problems in a suitable digraph even though they have nothing to do with routes and 
distances. In this section we explore some of the possibilities. 


Sequential Decision Problems 


One common feature of problems amenable to dynamic programming is sequential 
decision making—decisions that can be arranged in a clear sequence. Then the 
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decisions can be confronted one by one in sequence. That same sequence produces 
an acyclic digraph on which a shortest or longest path problem can be posed. 


EXAMPLE 9.5: WAGNER-WHITIN LOT SIZING 


A classic application, due to H. Wagner and T. Whitin, concerns lot sizing, that is, 
planning production in an environment where there is a substantial setup cost in- 
curred with each production run. ‘Table 9.9 details an example. For each time period 
k=1,...,n, we know 


r; & quantity of product required at time k 

sx & setup cost if production occurs at time k 

Px = variable, per unit cost of production at time k 
hy & unit cost of holding goods through period k 


[> 


TABLE 9.9 Wagner—Whitin Lot Sizing Data 


Period, k 
Wee eta O 
Requirement, r; 10 40 20 5 SeietlS 


Setup cost, 5, SO S050 S0e 0s 650 
Production cost, p, 1 3 3 1 1 1 
Holding cost, h;, 2 2 Dare) 2 2 


An optimal solution must carefully balance production and holding costs. Some- 
times it will be better to pay the setup cost and produce just as goods are required. 
Other times we will produce in one period for the next several, incurring a holding 
cost of storage and lost investment income on goods not required immediately. 


States in Dynamic Programming 


For our Wagner—Whitin inventory example the passage of time sequences the re- 
guired decisions. To form a dynamic programming model, we must characterize 
states of incomplete solution that capture the history on which each new decision 
should be based. 


-s in dynamic programmin 


ech 


If an optimal solution is known for any state, those of subsequent states can build 
upon it. 

Development of a state description for Wagner—Whitin lot sizing begins with a 
simplifying insight: production needs to occur in an optimal solution only when no 
inventory is held through the preceding period. That is, we should produce or hold 
for any period k, but not both. Any solution that combined holding and production 
could at least be matched by shifting the full 7, to whichever of unit holding cost 
since the last production and variable production cost p,; was lower. 
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The result is a definition of states. We may say that the process has reached state 
k when requirements for ones £ <k have been fulfilled and there is no on-hand 
inventory. 


Digraphs for Dynamic Programs 


A suitable definition of states begins the construction of the digraph underlying any 
elementary dynamic program. 


The remaining element is decisions. What options are available upon reaching 
any given state, and what state will we reach if a decision is adopted? 


decisions. They ao state —- sith a subset state to ae the decision. 
_ leads. - oo oo 


Figure 9.13 displays the digraph for our Wagner—Whitin lot-sizing example. 
Here decisions correspond to producing in some period to meet its requirements 
and, possibly, those of several subsequent periods. For example, the arc from state 
k =3 to £ = 6 reflects production at period 3 to meet requirements 73 = 20, r4 = 5. 
and rs = 5, arriving at period 6 with zero inventory. Its cost is 


C36 = setup + production + holding 
= 53 + (73 +174 + 1s) p3 + [(r4 + 175)h3 + (15)ha] 
= 50+ Q0-FS F5)3+ OF 5)24 @)Z 
= 170 


FIGURE 9.13 Digraph for Wagner—Whitin Example 


9.8 Discrete Dynamic Programming Models 


Dynamic Programming Solutions as an Optimal Path 


Notice now that each path from node 1 in the digraph of Figure 9.13 to node (n+1) = 7 
constitutes a lot-sizing plan. For example, path 1-3-6-7 corresponds to production 
in period 1 for requirements r; and rz, then in period 3 for requirements 73, r4, and 
rs, then in period 6 for r¢. The shortest such path identifies an optimal solution. 


9.39 IO Optimal solutions to elemen dynamic program ee to short- 


They can be solved by applying the appropriate optimal path algorithm. 

Like most dynamic programs, our lot-sizing problem is posed over an acyclic 
digraph because decision arcs at any state lead only to later states. Application of 
shortest acyclic Algorithm 9D produces the optimal solution highlighted in Figure 
9.13. It corresponds to producing in periods 1 and 5 at total cost of 415. 


E EXERCISE 9.19: FORMULATING DYNAMIC PROGRAMS 


The management team of a 5-year research project is planning a replacement policy 
for its personal computers. New models may be purchased for $3000 each. If sold 
after | year, they retain a salvage value of $1200. Ifsold after 2 years, the salvage value 
falls to $500, and after 3 years the units are obsolete and have no value. Maintenance 
costs also increase with age. They are estimated at $300 in the first year of service, 
$400 in the second year, and $500 in the third. Formulate a dynamic program and 
associated digraph to compute a minimum total cost replacement policy. 


Modeling: States in this problem correspond to years completed, and decisions relate 
to keeping computers for 1, 2, or 3 years. Cost of any decision is 


(purchase cost) — (salvage value) + (maintenance cost) 


The result is the following digraph: 


4200 4200 


An optimal solution corresponds to a shortest path from time 0 through year 5. An 
optimum is highlighted in the figure with total cost $7400. 


Dynamic Programming Functional Equations 


Although it is usually instructive to draw the digraph associated with a dynamic 
program, it is common to proceed directly to the underlying functional equation 
recursions. 
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nming functional equi on recursions encode connec- - 
tions among optimal values for different s of incomplete solution. — 


The process begins by interpreting the optimal value of any state. For example, 
with our Wagner—Whitin lot-sizing model, 


v[k] 4 minimum cost of fulfilling requirements for periods ¢ < k 
and arriving at k with no inventory 


Dynamic programming solution of the problem is then encapsulated in functional 
equation recursions 


v[1] = 0 
v[A] = min{v[€] +o, : 1 < € <k} k= 2.0. 
where 
Cok 2 Set (Tet + + 1K1)pe + (res +... Fea he He + Te) Ak-2 
For example, 
v[4] = min{v[1] aCe. v[2] + C24, v[3] + 3,4} 
min{v[1] + 280, v[2] + 270, v[3] + 110} 


These are precisely the shortest path functional equations of definition for the 
digraph of Figure 9.13. 


| SAMPLE EXERCISE 9.20: FORMULATING DYNAMIC PROGRAM RECURSIONS 


Formulate functional equations for the dynamic programming model of Sample 
Exercise 9.19. 


Modeling: For this model 
v[k] & minimum cost of providing computers through year k 
This leads to functional equations 
v[0] = 0 
v[k] = min{v[k —3]+c3, v[k —2]+o, v[k—-1]+c} Klas oD 


where c; is the total cost of keeping a computer for j years. 


Dynamic Programming Models with Both Stages and States 


A sequence of decisions based on states of incomplete solution lies at the heart of 
all discrete dynamic programs. However, it is often convenient to distinguish stages 
of decision making from states of solution. 


decisions can be considered. 


9.8 Discrete Dynamic Programming Models 


EXAMPLE 9.6: PRESIDENT’S LIBRARY 


We may illustrate with the design of shelving in the president’s library.! The retiring 
president is collecting all his papers in a new presidential library. Materials for its 
archives are stored in covered cardboard boxes. All are 1.25 feet wide, but their 
height varies. Table 9.10 shows the estimated number of boxes at each height. 


TABLE 9.10 President’s Library Storage Requirements 


i 1 74 3 4 5 6 7 
Height in feet, h; 0.25 0.40 0.80 OOPS VSO) 2200/5 = 3:0 
Thousands of boxes,b; — 10 2 12; 30 8 6 4 


Boxes in the archives will be placed on metal shelves, with no box stacked on 
top of another. The issue is how much shelving will be required. Figure 9.14 shows 
that if, for example, two shelf spacings are employed, the total face area of shelving 


is 


linear feet of iY linear feet of 1 eee 
boxes with shelf a boxes with ioe 
height < smaller a height > smaller eset 
shelf spacing P 8 shelf spacing 8 
box 
cs area for second spacing o 
A 

eye posssene 

1 

1 

1 

1 

1 

1 

CY eee 
ike I 

I 

1 

1 


area for first otay 


feet of 


oe il, shelves 
20 30 40 50 60 70 (000s) 


10 
FIGURE 9.14 President's Library Shelving Face Areas 
The most compact shelving obviously will result if different spacings are pro- 


vided for each of the 7 sizes in Table 9.10. Still, designers would like to keep storage 
relatively uniform. Whether they choose to provide 1, 2,...,7 spacings, they want 


to use the space as efficiently as possible. 


‘Based in part on FE F Leimkuhler and J. G. Cox (1964), “Compact Book Storage in Libraries,” 
perations Research, 12, 419-427. 
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Dynamic Programming Modeling of the President’s 
Library Example 


To distinguish stages for a dynamic programming model of the president’s library 
example, we must consider the sequence of decisions. Here decisions correspond to 
choices of shelf spacing. Thus we will have one stage for each distinct spacing. More 
precisely, 


stage k 4 kth-to-last choice of a spacing 


Next come states. On what preconditions should decisions be based? States 
are often the least intuitive part of forming a dynamic programming model. In our 
president’s library example, the decision to allocate a new shelf spacing depends on 
what box sizes have already been provided for. Thus we define states 


state i A having provided shelves for box sizes 0,1, ...,i (0 indicates none) 


Decisions, which constitute the third element of any dynamic programming 
model, are now easy to describe. The choice to provide a spacing for box sizes 
through j, given that those through i are already accommodated, means commitment 
of shelf face area 


Gp 125 ( ye a 
i<s<j 


For example, accommodating box sizes through j = 6, given that those up i = 3 have 
already been provided for, implies an area of 


1.25(b4 + bs + be)he 
1.25(30 + 8 + 6)(2.0) = 110.0 


C3,6 


thousand square feet. 
Figure 9.15 shows the digraph for this dynamic program. Notice that there is 
one group of nodes for each stage. In accord with principle 9.37], nodes distin- 


guish states. Arcs of the digraph reflect decisions (principle ). For example, 
the arc from node 3 of stage 2 to node 6 of stage 1 corresponds to making the 
second-to-last spacing for boxes accommodate sizes 4 to 6. All arcs from the last 
stage lead to an artificial finish node that corresponds to having provided for all box 
sizes. 


An entrepreneur with $8 million to invest is considering acquisition of four com- 
panies. She estimates the cost of acquiring the four at $5, $1, $2, and $7 million. 
respectively, and the present value of owning them at $8, $3, $4, and $10. In- 
vestments are on an “all or nothing” basis (i.e., she must buy entire companies). 
Model as a dynamic program the problem of deciding which investments she should 
choose. 


9.8 Discrete Dynamic Programming Models 


stage stage stage stage stage stage stage stage 
e=7 k=6 =) k=4 k=3 k=2 kad k=0 


Figure 9.15 Digraph for President’s Library Example 


Modeling: It is clear that decisions concern whether our entrepreneur should in- 
vest in each of the 4 companies. Thus we associate stages with companies k = 
12.3, 4. 

For states we must decide what information about prior decisions is needed 
to know whether a current one is possible. Here we need to know the amount of 
investment money remaining when the decision stage for a particular company is 
reached. The result is statesi = 0,1,...,8. 

At each stage & and state i, two decisions are possible. The entrepreneur can 
choose not to buy the kth company and pass along amount i, or she can buy the kth 
company and advance to state (i — ax), where a, & purchase price of company k. Of 
course, the latter option is available only when i > ax. 

Denote 


v,{i] & maximum total present worth obtainable from 
companies £ > k given that i million dollars is 
available for investment 


Then functional equations are maximize versions of format (9.1): 
vx[i] = 0 k>4ori<0O 
vgfi] = max{vgsi[Z], ce + vias li — ax]} Kevan Ao ard Oo 


where c, £ present worth of company k. The associated digraph is as follows: 
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stage 1 stage 2 


The highlighted optimal solution corresponds to a longest path from state 8 of stage 
1 to finish. 


Backward Solution of Dynamic Programs 


Readers may wonder why we have chosen to number stages of the president's li- 
brary example in reverse or backward sequence. This is to highlight the sequence of 
solution we will apply. 


Then optimal solutions correspond to shortest or longest paths to the final state in 
an associated digraph. 

The president’s library example is no exception to principle . Its func- 
tional equations take the form 


optimal cost 
opinealecst immediate to finish 
( ‘ofa ) = min decision + | from the (9.1) 
cost resulting 
state 


where the minimization is over the available decisions. More precisely, define 


vi [i] & optimal cost to finish from stage k, state i 


v7[7] =1() 


vz[i] = min{¢;; + Ve-1 [J] ya} 


9.8 Discrete Dynamic Programming Models 


ie = Neabs6stn asses b) 


An optimal solution now corresponds to a shortest path to finish rather than the 
ssual shortest path from a single source. However, Algorithm 9D is easily adapted. 
ve need only evaluate the v;[£] in reverse acyclic sequence. 

Table 9.11 processing begins by setting the cost of the final state Vanish < 0. 
then all stage k = 1 states are solved in turn, each with only one out arc leading to 


Snish. 


A more typical step arises if fixing v»[3]. This second-to-last shelf spacing (given 
»oxes through size 3 are handled) could provide for size 4, or sizes 4 and 5, or sizes 


+to6. Thus 


v2[3] <— min{c34 +14 [4], 35+ [5], C36 + VY [6]} 


ll 


min{37.5 + 67.5, 71.25 + 37.5, 110.0 + 15.0} 
min{105.0, 108.75, 125.0} = 105.0 


thousand square feet, with the optimal decision leading to state 4. 


TABLE 9.11 Backward Solution 


of President’s Library Example 


Stage, 


finish 
1 


State, 
i 


Optimal Area, 
Vili] 
0.000 


270.000 
232.500 
225.000 
180.000 
67.500 
37.500 
15.000 


Next State, 
d,{i] 


135.000 
122.500 
120.000 
105.000 
50.000 
30.000 


117.500 
105.000 
102.500 
87.500 
45.000 


108.125 
100.000 
97.500 
82.500 


103.125 
96.500 
94.500 


99.625 
95.500 


98.625 


PIP IWWrPIARARI AAR ER DE DAFAAAIYAIYAYI YIN] CO 
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Multiple Problem Solutions Obtained Simultaneously 


What solution is optimal in Table 9.11? There are several, depending on how many 
distinct shelf spacings are adopted. Optimal shelf areas are 


v1 [0] = 270.0 
v»[0] = 135.0 
v3[0] = 117.5 


[ 
3 
v4[0] = 108.125 
ys[0] = 103.125 
v6[0] = 99.625 
v7[0] = 98.625 


for k = 1 spacing 

for k = 2 spacings 
for k = 3 spacings 
for k = 4 spacings 
for k = 5 spacings 
for k = 6 spacings 
for k = 7 spacings 


with corresponding optimal solutions retrievable from corresponding d;,[é] labels. 
Solutions for multiple assumptions are a common benefit of dynamic program- 


ming solution. 


9.43 | The dynamic programming strategy of exploiting connections among op- 
timal solutions for a number of related problems often implies that optima for 


multiple problem scenarios are produced by a single shortest or longest path 


computation. 


EXERCISES 
9-1 Consider the following graph. 


10 4 


Numbers on arcs and edges represent lengths. 


(a) [I Identify the nodes, arcs, and edges of the 
graph. 

(b) [S| Determine whether each of the following se- 
quences is a path of the graph: 1-3-4-5, 2-5-3- 
4, 1-3-2-5—-4, 1-3-4-1-2. 

(c) [Q] Direct the graph (i-e., exhibit a digraph with 
the same paths as those of the given graph). 


9-2 Do Exercise 9-1 for the graph 


and sequences 3—2-1-3-5, 3-2-4, 1-3-2-4-5, 1-3-5- 
D. 


9-3 Return to the problem of Exercise 9-1. 


(a) K] Find (by inspection) shortest paths from 
node 1 to all other nodes. 

(b) [| Verify that every subpath of the optimal 1 to 
2 path in part (a) is itself optimal. 

(c) KI Detail your optimal solutions of part (a) in 
functional notation v[k] and x;;[k]. 

(d) EQ] Write functional equations for the shortest 
path problems of part (a). 


e) Verify that the v[k] of part (c) satisfy functional 
equations of part (d). 

f) [6] Explain why functional equations are suffi- 
cient to characterize optimal values v[k] for this 
instance. 


8-4 Do Exercise 9-3 for the problem of Exercise 9-2, 
verifying the optimal 1 to 3 path in part (b). 
8-5 Consider the following graph. 


a) [9] Find (by inspection) shortest paths from all 
nodes to all other nodes. 


5) [&] Verify that every subpath of the optimal 1 to 
4 path in part (a) is itself optimal. 

ce) [| Detail your optimal solutions of part (a) in 
functional notation v[k][£] and x;;|k]|é]. 


4) [| Write functional equations for the shortest 
path problems of part (a). 

=) Verify that the v[k][@] of part (c) satisfy func- 
tional equations of part (d). 

*) [Q Explain why functional equations are suffi- 
cient to characterize optimal values v[k][é] for 
this instance. 


8-6 Do Exercise 9-5 for the following graph, verify- 
ing the optimal 4 to 2 path in part (b). 


9-7 Consider the following digraph: 


469 


Exercises 


Numbers on arcs represent costs. 


(a) [Q] Find (by inspection) shortest paths from 
node 1| to all other nodes. 

(b) [&] Enumerate the dicycles of the graph. 

(c) Determine whether each dicycle is a nega- 
tive dicycle. 

(d) EQ] How does the presence of negative dicycles 
in this instance make it more difficult for an al- 
gorithm to compute shortest paths? 


9-8 Do Exercise 9-7 for the digraph 


9-9 Return to the graph of Exercise 9-1, and suppose 
that we seek shortest paths from node 1 to all other 
nodes. 


(a) [| Explain why Bellman-Form Algorithm 9A 
can be employed to compute the required short- 
est paths. 

(b) [| Apply Algorithm 9A to compute the lengths 
of shortest paths from node 1 to all other nodes. 

(c) [Q] Use d[k] labels of your computations in part 
(b) to recover all optimal paths. 

(d) [| Verify the interpretation of interim labels 
v[k] as lengths of shortest paths using at most 
t arcs/edges by showing that values at the end of 
iteration ¢ = 2 in your computations of part (b) 
correspond to lengths of shortest paths using 1 
or 2 arcs/edges. 
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(c) [Q] Determine the maximum number of 
arcs/edges in any path of this graph, and explain 
how this bounds the computation required for 
Algorithm 9A. 


9-10 Do Exercise 9-9 on the graph of Exercise 9-2. 


9-11 Use Bellman-Ford Algorithm 9A to identify a 
negative dicycle in each of the following graphs. 


(a) [| The digraph of Exercise 9-7 
(b) The digraph of Exercise 9-8 


9-12 Return to the graph of Exercise 9-5, and sup- 
pose that we seek shortest paths from all nodes to all 
other nodes. 


(a) [| Explain why Floyd—Warshall Algorithm 9B 
can be employed to compute the required short- 
est paths. 

(b) [I] Apply Algorithm 9B to compute the length 
of shortest paths from all nodes to all other 
nodes. 

(c) [S| Use d[k][é] labels of your computations in 
part (b) to recover all optimal paths. 

(d) [| Verify the interpretation of interim labels 
v[k][é] as lengths of shortest paths using only 
intermediate nodes up to ¢ by showing that val- 
ues at the end of iteration ¢ = 2 in your com- 
putations of part (b) correspond to lengths of 
shortest paths using only nodes 1 and 2 as inter- 
mediates. 


9-13 Do Excercise 9-12 on the graph of Exercise 9-6. 


9-14 Use Floyd—Warshall Algorithm 9B to identify 
a negative dicycle in each of the following graphs. 


(a) [Q] The digraph of Exercise 9-7 
(b) The digraph of Exercise 9-8 


9-15 Return to the graph of Exercise 9-1, and sup- 
pose that we seek shortest paths from node 1 to all 
other nodes. 


(a) | Explain why Dijkstra Algorithm 9C can 
be employed to compute the required shortest 
paths. 

(b) [| Apply Algorithm 9C to compute the length 
of shortest paths from node 1 to all other nodes. 

(c) EQ] Use d[k] labels of your computations in part 
(b) to recover all optimal paths. 

(d) [9 Verify the interpretation of interim labels 
v[k] as lengths of shortest paths using only per- 
manently labeled nodes by showing that values 
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after two permanent nodes have been processed 
in your computations of part (b) correspond to 
lengths of shortest paths using only those nodes 
(and the destination). 


9-16 Do Exercise 9-15 on the graph of Exercise 9-2. 
9-17 For each of the following digraphs, deter- 
mine which of Bellman-Ford Algorithms 9A, Floyd— 
Warshall Algorithm 9B, and Dijkstra Algorithm 9C 
could be applied to compute shortest paths from 
node 1 to all others. Then, if any is applicable, choose 
the most efficient and apply it to compute the re- 
quired paths. 


(a) & 


(b) 


(d) 


9-18 Demonstrate whether each of the following di- 
graphs is acyclic by exhibiting a dicycle or numbering 
nodes so that every arc (i,j) has i <j. 


9-19 Consider the following digraph: 


Suppose that labels on arcs are cost and that we seek 
a minimum total cost path from node 1 to all other 
nodes. 
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Exercises 

(a) Demonstrate that acyclic shortest path Al- 

gorithm 9D can be applied to compute the re- 
quired paths. 

(b) Use Algorithm 9D to compute the lengths 

of shortest paths from node 1 to all other nodes. 


(c) Use d[k] labels from your computations in 
part (b) to recover all optimal paths. 


9-20 Do Exercise 9-19 for the digraph 


9-21 Determine whether acyclic shortest path Al- 
gorithm 9D could be applied to compute shortest 
paths from node 1 to all other nodes in each digraph 
of Exercise 9-17. If so, explain whether it would 
be more efficient than the Bellman—Ford, Floyd— 
Warshall, and Dijkstra alternatives, and why. 


9-22 The table that follows lists the tasks that must be 
performed in preparing a simple breakfast. The ta- 
ble also shows the number of minutes each requires 
and the tasks that must be completed before each 
can begin. 


k Task Time _ Predecessors 
1 Boil water a None 

2 Get dishes il None 

3. Make tea 3 ED; 

4 Pour cereal 1 2 

5 Fruit on cereal 2 4 

6 Milk on cereal 1 4 

7 Make toast 4 None 

8 Butter toast 3) I 


(a) [&] Construct the corresponding CPM project 
network | 9.30 |. 

(b) [S| Use given activity numbers to verify that 
your project network is acyclic. 

(c) | Apply CPM scheduling Algorithm 9E to 
compute early start times for each activity and 
an early finish time for the entire project. 

(d) [| Use d[k] labels of your scheduling compu- 
tations to identify the activities along a critical 
path from project start to finish. 

(e) EK] Compute late start times for each activity as- 
suming that the breakfast must be complete in 
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10 minutes, and combine with part (c) to deter- 
mine schedule slacks. 


9-23 The table that follows lists the activities that 
must be performed in organizing a soccer tourna- 
ment. The table also shows the estimated number 
of days that each will require and the activities that 
must be completed before each can begin. 


k Activity Time Predecessors 
1 Select dates il None 

2 Recruit sponsors 4 1 

3 Set fee if 2, 

4 Buy souvenirs 5) V2 

5 Mail invitations 1 253) 

6 Wait responses 4 5) 

7 Plan pairings il 6 

8 Team packets il} 4,7 


(a) through (e) as in Exercise 9-22, assuming that 
the organization must be completed in 13 days. 


9-24 Construction of a small two-story house in- 
volves the tasks listed in the following table. The 
table also shows the estimated duration of each task 
in days and the tasks that must complete before it 
can begin. 


Task Time _Predecessors 
Foundation (FD) 8 None 
Concrete slab (CS) 5 FD 
First bearing walls (1B) 3 CS 
First internal walls (11) 4 GS 
First finishing (1F) 12 1B, 11, FL 
Second floor (FL) 3 1B 
Second bearing walls (2B) 4 2FL 
Second internal walls (21) 5 2FL 
Second finishing (2F) 10 2B, 21,R 
Roof (RF) 2 2B 


(a) [| Construct the corresponding CPM project 


network ‘ 


(b) [] Number activity nodes in your project net- 
work so that every arc (i,j) has i <j. 

(c) [>] Use CPM scheduling Algorithm 9E with 
your activity numbering to compute early start 
times for each activity and an early finish time 
for the entire project. 

(d) [Q] Use d[k] labels of your scheduling compu- 
tations to identify the activities along a critical 
path from project start to finish. 

(ec) [I] Compute late start times for each activity 
assuming that construction must be complete in 
35 days, and combine with part (c) to determine 
schedule slacks. 
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9-25 The table that follows shows the activities re- 
quired to construct anew computer laboratory, along 
with their estimated durations (in weeks) and prede- 
cessor activities. 


Activity Time _Predecessors 

Order furniture (OF) 1 None 
Order computers (OC) 1 None 
Order software (OS) 1 Oc 
Furniture delivery (FD) 6 OF 
Computer delivery (CD) 3 oe 
Software delivery (SD) 2 OS 
Assemble furniture (AF) 1 P, FD 
Install computers (IC) 1 AF, CD 
Install software (IS) it IC, SD 
Wire room (W) 2 None 
Paint room (P) 1 Ww 


(a) through (e) as in Exercise 9-24, assuming that 
the lab must be completed in 15 weeks. 


9-26 A pharmaceutical manufacturer must supply 
30 batches of its new medication in the next quarter, 
then 25, 10, and 35 in successive quarters. Each quar- 
ter in which the company makes product requires 
a $100,000 setup, plus $3000 per batch produced. 
There is no limit on production capacity. Batches 
can be held in inventory, but the cost is a high $5000 
per batch per quarter. The company seeks a mini- 
mum total cost production plan. 


(a) Explain why this problem can be approached by 
dynamic programming, with statesk =1,...,5 
representing the reaching of quarter k with all 
earlier demand fulfilled and no inventory on 
hand. 

(b) [| Sketch the digraph corresponding to the dy- 
namic program structure of part (a). Include 
costs on all arcs. 

(c) Explain why the feasible production plans cor- 
respond exactly to the paths from node k = 1 
to node k = 5 in your digraph. 

(d) [] Solve a shortest path problem on your di- 
graph to compute an optimal production plan. 

(e) &] Use your computations in part (d) to com- 
pute an optimal production plan for the first two 
quarters. 


9-27 Do Exercise 9-26 with all parameters the same 
except a holding cost of $2000. 


9-28 A copy machine repairman has four pieces of 
test equipment for which he estimates 25%, 30%, 
55%, and 15% chances of using them at his next 


stop. However, the devices weigh 20, 30, 40, and 20 
pounds, respectively, and he can carry no more than 
60 pounds. The repairman seeks a maximum utility 
feasible collection of devices to carry with him. 


(a) Explain why this problem can be approached by 
dynamic programming, with stagesk =1,...,4 
representing the four devices and states w = 
0, 10, 20, 30, 40, 50, 60 in each stage indicating 
reaching that decision stage with w units of 
weight limit remaining. 

(b) [§] Sketch the digraph corresponding to the dy- 
namic program structure of part (a). Include 
objective function contributions on all arcs. 

(c) Explain why the feasible production plans cor- 
respond exactly to the paths from stage k = 1, 
state w = 60 to the last stage in your digraph. 

(d) [S| Solve a longest path problem on your di- 
graph to compute an optimal toolkit. 


9-29 Do Exercise 9-28 with all parameters the same 
except a weight limit of 50 pounds. 


9-30 The figure that follows shows a partially com- 
plete layout for a circuit board of a large computer 
peripheral. Lines show channels along which circuits 
can be placed, together with the lengths of the chan- 
nels in centimeters. Several circuits can be placed in 
a single channel (on different layers). 


27 


The last step in the design is to choose the routing 
from a component to be installed at point 1 to con- 
nections at points 8, 10, 11, and 12. 


SSS Ss Sel en a earner 
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Exercises 


(a) Explain why this problem can be modeled as a 
shortest path problem. 

(b) Explain why the most efficient procedure avail- 
able from this chapter for computing optimal 
circuit routes is Dijkstra Algorithm 9C. 

(c) Apply Algorithm 9C to compute optimal cir- 
cuit routes to the 4 specified points. 


9-31 The figure that follows shows the links of a pro- 
posed campus computer network. Each node is a 
computer, and links are fiber-optic cable. 


11 25 
10) 
23 
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z 
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Designers now want to decide how E-mail, which 
will be broken into standard-length packets, should 
be routed from internet gateway node 1 to/from all 
other nodes. For example, E-mail for node 4 might 
be transmitted by 1 to 6, then repeated by 6 to 5, the 
repeated by 5 to4. Numbers on nodes in the figure in- 
dicate minimum times (in nanoseconds) required by 
the corresponding computer to transmit or receive 
a message packet. The time to send a packet along 
any link of the network is the maximum of the times 
for the associated sending and receiving computers. 


(a) Explain why this problem can be modeled as a 
shortest path problem, and sketch the graph and 
edge weights over which optimal paths are to be 
computed. 

(b) Explain why the most efficient procedure avail- 
able from this chapter for computing optimal 
E-mail routes is Dijkstra Algorithm 9C. 

(c) Apply Algorithm 9C to compute optimal 
routes to all computers. 


9-32 The campus shuttle bus begins running at 7:00 
PM. and continues until 2 A.M. Several drivers will be 
used, but only one should be on duty at any time. Ifa 
shift starts at or before 9:00 PM., a regular driver can 
be obtained for a 4-hour shift at cost $50. Otherwise. 
part-time drivers will be used. Several would work 
3-hour shifts at $40 and the rest are limited to 2-hour 
shifts at $30. 
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(a) Showthatthe problem of computing a minimum 
total cost nightly driver shift schedule can be 
modeled as a shortest path problem on a graph 
with nodes equal to the hours of the night from 
7:00 PM. through 2 A.M. Also sketch the corre- 
sponding digraph, and label it with arc lengths. 

(b) [§] Is your digraph acyclic? Explain. 

(c) Determine which of the algorithms of this chap- 
ter would solve your shortest path problem most 
efficiently, and justify your choice. 

(ad) [@] Apply your chosen algorithm to compute an 
optimal nightly shift schedule. 


9-33 The machine shop depicted in the figure that 
follows has a heat treatment workstation at point 1, 
forges at points 2 and 3, machining centers at points 
4, 5, and 6, and a grinding machine at 7. Each grid 
square indicated is the same size. 


Processing of a camshaft product begins with heat 
treatment, then goes to any forge, then moves to any 
machining center, and finishes at the grinding ma- 
chine. Movement between workstations is rectilin- 
ear (i.c., north/south displacement plus east/west). 


(a) Show that the problem of computing an mini- 
mum total movement canshaft routing can be 
modeled as a shortest path problem. Also 
sketch the corresponding digraph, and label it 
with arc lengths. 

(b) Is your digraph acyclic? Explain. 

(c) Determine which of the algorithms of this chap- 
ter would solve your shortest path problem most 
efficiently, and justify your choice. 
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(d) [| Apply your chosen algorithm to compute an 
optimal movement. 


9-34 The figure that follows shows the trails of Lit- 
tleville’s Memorial Park. 


Whenever there is a major event there, heavy hot dog 
and soft-drink carts stored in the shed and rolled out 
along the trails to the five marked locations. Num- 
bers on trail links indicate their length, and arrows 
show which way is uphill. The work to push a cart up- 
hill on any link is proportional to that length. Carts 
can also be moved in the opposite direction, but the 
downhill effort is only half as much. 


(a) Show how the problem of finding the Icast total- 
effort-push routes for all carts can be modeled 
as a shortest path problem. Also sketch the cor- 
responding digraph, and label it with arc lengths. 

(b) Is your digraph acyclic? Explain. 

(c) Determine which of the algorithms of this chap- 
ter would solve your shortest path problem most 
efficiently, and justify your choice. 

(d) [| Apply your chosen algorithm to compute 
optimal move routes. 


Denardo, E. V. (1982), Dynamic Programming: 
Models and Applications, Prentice Hall, Upper Sad- 
dle River, N.J. 
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We have seen in Chapters 4 to 7 that linear program models admit some very ele- 
eant analysis. Global optima can be computed efficiently, and all sorts of “what if” 
sensitivity analysis can be performed on results. 

Network flow problems are special, yet widely applicable cases of linear pro- 
erams which prove even more tractable. Much larger models can be solved, because 
specialized algorithms apply. Most important, discrete cases, which we know are 
usually more difficult, can often be managed with no extra effort at all. 


19,1 GRAPHS, NETWORKS, AND FLOWS 


Dne of the things that make network flow models particularly tractable is that 
decisions and constraints have a form that we can easily represent in a diagram. 
More precisely, network flow models arise on structures called directed graphs 
or digraphs. 


Digraphs, Nodes, and Arcs 


We encountered digraphs in Section 9.1. They begin with a collection of nodes (or 
yertices), which we denote throughout this chapter by 


V A {nodes or vertices of the network} 


These indicate the facilities, or intersections, or transfer points of the network. Nodes 
are joined in digraphs bya collection of ares, which we denote throughout this chapter 
by 

A 4 {arcs of the network} 


Arcs show possible flows or movements from one node to another. We indicate in- 
dividual arcs simply by listing the pair of nodes they connect. For example, arc (4, 7) 
would go from node 4 to node 7. 

Digraphs are termed directed because the direction of flow matters. For exam- 
ple, an arc leading from node 7 to node 4, which would be denoted (7, 4), is not the 
same as (4, 7). They represent traffic in different directions. 
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EXAMPLE 10.1: OPTIMAL OVENS (OOJT) 


As usual, it will be much easier to absorb key notions with a small model in mind. 
Consider the (entirely ficticious) case of Optimal Ovens, Incorporated (OOI). 

OOI makes home toaster ovens at plants in Wisconsin and Alabama. Com- 
pleted ovens are shipped by rail to one of OOI’s two warehouses in Memphis and 
Pittsburgh, and then distributed to customer facilities in Fresno, Peoria, and Newark. 
The two warehouses can also transfer small quantities of ovens between themselves. 
using company trucks. 

Our task is to plan OOI’s distribution of new model E27 ovens over the next 
month. Each plant can ship up to 1000 units during this period, and none are presently 
stored at warehouses. Fresno, Peoria, and Newark customers require 450, 500, and 
610 ovens, respectively. Transfers between the warehouses are limited to 25 ovens. 
but no cost is charged. Unit costs (in dollars) of other possible flows are detailed in 
the following tables. 


From/To 3:Memphis 4: Pittsburgh 
1: Wisconsin 7 8 
2: Alabama 4 7 
From/To 5: Fresno 6:Peoria 7: Newark 
3: Memphis ps) 5 17 
4: Pittsburgh 29 8 5 
OOI Example Network 


Figure 10.1 depicts OOI’s network. The 2 plants, 2 warehouses and 3 customer sites 
make up the 7 nodes of this digraph. Arcs show the possible oven flows, with arrow- 
heads indicating direction. For example, arc (3, 7) denotes ovens shipped from the 
Memphis warehouse to the Newark customer site. The absence of an arc (7, 3) means 
that ovens are not allowed to make the opposite move from Newark to Memphis. 
Two opposed arcs joining the warehouse nodes, indicate that traffic between them 
can flow in either direction. 


Fresno 


Wisconsin Memphis 


Alabama Pittsburgh 1 
FIGURE 10.1 Network for Optimal Ovens, 


Newark Incorporated (OOI) Example 


10.1. Graphs, Networks, and Flows 


Minimum Cost Flow Models 


What linear program doesa digraph like Figure 10.1 represent? We want it to describe 
flows, here of ovens starting at plants, passing through warehouses and terminating 
at customers. 

As usual, we begin with decision variables. 


in arcs (7, )). 
Letting c;; denote the unit cost of flow on arc (i, j), the total cost to be minimized 


is simply 
> Ci jXij 
(ieA 
Some constraints are equally easy. Flows must be nonnegative to make sense, 
and capacities or upper bounds u;,; may apply. These requirements lead to con- 
straints: 


0 < xij < Ui; for all (i,j) € A (10.1) 
The defining characteristic of network flow problems is the form of their main 
constraints. 


10.2_| Main constraints of network flow problems enforce balance (OF e¢ conser- 
vation) of flow at nodes, 


More precisely, we want 
(total flow in) — (total flow out) = specified net demand 
at every node. In symbols, 


) Xe Do, Mey = b, forallke V (10.2) 
(.KeEA (k.eA 
where b; denotes the specified net demand (required flow inbalance) at node k. 
We are now ready to state the full minimum cost network flow model form: 


min = Ge : : eee = : 
(ped — - — ee 
st. > Xik = >| ‘yj = OK for all k oe 
Gee le 
Oe For all pea A 


Sources, Sinks, and Transshipment Nodes 


Nodes come in three types. Sink or demand nodes such as the customer sites in the 
OOI example consume flow. Source or supply nodes such as the OOI plants create 
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flow. Transshipment nodes such as OOI warehouses merely pass along flow. Net 


demands b; have corresponding sign. 


OOI Example Model 


Figure 10.2 includes net demands, costs and capacities on the digraph for our OOI 
example (plus an extra node 8 explained below). The corresponding minimum cost 


network flow model is 
min 7x1,3 + 8x1,4 + 4x23 + 7x2,4 + 25x35 + 5x3,6 
+ 17x37 + 29x45 + 8x46 + 5x47 


St. —%1,3 — X1,4 — X18 = —1000 
— X2,3 — X2,4 — X28 = —1000 
X13 1 003 ds Had ss hog a 0 
+4%1,4 + X2,4 + 3,4 — X43 — X45 — X46 — X47 = (0) 
+ X3,5 + X4,5 = 450 
+ X3,6 + X4,6 === 00) 
+ %3,7 + X4,7 =. 60 
+ %1,8 + X2,8 = 440 


%3,4 =< 25, X43 < 25 
xij 20 forall @j eA 


Heavy lines in Figure 10.3 show an optimal solution. 


(c, 4) ———_ 


Fresno 


Wisconsin / Memphis > (5) 450 


(7, 2) 


(total cost) 
(node 1) 
(node 2) 
(node 3) 
(node 4) 
(node 5) 
(node 6) 
(node 7) 
(node 8) 
(capacities) 


(10.3) 


i=) 
Peoria 
440 6) 500 


Alabama Pittsburgh @ 610 


Newark 


FIGURE 10.2. Minimum Cost Network Flow Problem 
for OOI Example 


10.1 Graphs, Networks, and Flows 


Ae Fresno 
G) 450 


Wisconsin Memphis 450 


Alabama Pittsburgh ep 
610 
Newark 
FIGURE 10.3 Optimal Flows for OOI Example 


Plants are the source nodes of the OOI network. Thus their flow balance con- 
straints (the first two) in model (10.3) have negative net demands. Warehouse nodes 
merely transship flow, so net demand is zero. The last four balance constraints of 
model (10.3) detail the sink nodes consuming flow; corresponding right-hand sides 
are positive. 

The OOI example has capacities only on arcs connecting the two warehouses. 
No more than 25 ovens may be transferred in either direction. Bound constraints in 
model (10.3) reflect these two capacity limits and nonnegativity on all arcs. 


1: FORMULATING MINIMUM Cost NETWORK FLOW 
MODELS _ = — 


(a) Formulate the corresponding minimum cost network flow problem. 
(b) Classify nodes of the problem as source, sink, or transshipment. 
Modeling: Here 

Ve=n(ils 2354} 

A = {(, 2), A, 4), @, 3), 2,4), G, 2), 4, 3} 
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(a) Using variables x12, *1,4, 2,3, X2,4, X4,2, and x43 to represent flows on the six 
members of A, the formulation of principle i: 


min 2x12+3x14+ 5x2,3 — 1x42 + 11x43 


s.t. Ki oped = —100 
X12 + X42 — X2,3 — X2,4 = 0 
X2,3 + X43 = 60 
X14 +%2,4 — X42 — X43 = 40 


X12 < 90, x14 < 75, x23 < 50 
xij 20 foral@jeA 


(b) The only node with a negative net demand or supply is node 1. Thus it is 
the only source node. Nodes 3 and 4 have positive net demand, making them 
sinks. Remaining node 2, which has neither a demand nor a supply, is a transship- 
ment node. 


Total Supply = Total Demand 
One element appears in Figure 10.2 and model (10.3) that was absent in original 
Figure 10.1. An extra sink node 8 has been added. 

To see why, look again at flow balance constraints of . Flow is created 
only at source nodes and consumed only at sink nodes. Thus there is no hope of a 
feasible flow unless 

total supply = total demand 

which means )°, by = 0. 

When the given network of a minimum cost network flow problem does not 


come to us with total supply equal total demand, we must make some adjustments 
before proceeding. 


sources. oe = ee 


Node 8 in Figure 10.2 was added because 
total supply = 1000 + 1000 > 450 + 500 + 610 = total demand 


in the OOI example. The excess of 440 determines node 8’s demand. We want 
this excess supply to be able to reach node 8 without affecting any other part of the 
optimization. Zero-cost arcs from both source nodes do the job. 


The Aeeph below Sheaae net Head we next to each of its nodes aa cost cj; On 
its arcs. 


10.1 Graphs, Networks, and Flows 


Modify the network as required to produce an equivalent one with total supply equal 
to total demand. 


Vodeling: Here total supply is 50 + 90 = 140 and total demand is 28 + 52 = 80. 
Thus total supply exceeds total demand by 
140 — 80 = 60 


To produce an equivalent model with total supply equal to total demand, we 
apply principle and add a “dummy” sink 5 to consume the excess. 


Zero-cost arcs (1,5) and (3, 5) from the two sources assure that the extra supply can 
reach node 5 without affecting other costs. 


Node—Arc Incidence Matrices and Matrix Standard Form 


As in all our work with linear programs, we will often wish to think of minimum cost 


network flow model| 10.3 | in matrix terms. To do so, we will abuse notation to treat 
the flow variables x;; as a vector x even though components have two subscripts. 
Then, collecting costs c;; and capacities u;; in corresponding vectors ¢ and u, and 
arraying net demands by, in vector b reduces the minimum cost network flow model 


10.3 | to the familiar LP standard form 


min c:x 
st. Ax=b (10.4) 
0O<x<u 


Main constraint matrix A has a very special structure. Such matrices are called 
node-arcincidence matrices because they both encode the flow balance requirements 
and provide an algebraic description of the underlying digraph. 
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ATC. The only nonzero entries in each column are 
i the corresponding arc leaves and a +1 in the row 
for the node the arc enters. — : 


Table 10.1 illustrates for the OOI example. The 8 rows correspond to the 8 
nodes of the digraph in Figure 10.2. One column is present for each of the 14 arcs. 
The column for arc (3, 6) has a —1 in row 3 and a +1 in row 6 because arc (3, 6) leaves 
3 and enters 6. 

One of the conveniences of network flow problems is that we can depict them 
in network diagrams. Notice, however, that we could just as well start from the node— 
arc incidence matrix. If we had been given Table 10.1, it would be easy to sketch the 
corresponding digraph. 


Consterist the node-are moidence n matrix tet the oaenel esi of Sample Exercise — 
10.2. 
Modeling: Consistent with principle , the node—arc incidence matrix will have 


a row for each of the 4 nodes and a column for each of the 5 arcs. The full matrix is 
as follows: 


Are 
Node (1,2) (1,4) (2,4) (3,1) (4,3) 
1 -1 -1 0 +1 0 
2D; +1 0 -1 0 0 
3 0 0 0 -1 +1 
4 0 +1 +1 0 -1 


Conger the siren matrix: 


silat O +1 0 0 
0 -1 -1 0 O +41 
0 oO +1 -1 -1 0 

+1 0 0 OL atl ill 


(a) Explain why it is a node—arc incidence matrix. 
(b) Draw the corresponding digraph. 
Modeling: Again we apply principle : 


(a) This is a node—arc incidence matrix because each column has only two nonzero 
entries, one —1 and one +1. 
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TaBLE 10.1 Node-Arc Incidence Matrix of the OOI Example 


Are 
Nede (1,3) (1,4) (1,8) (2,3) (2,4) (2,8) (3,4) (,5) (3,6) (3,7) (4,3) (4,5) (4,6) (4,7) 
: -l -1 -1 0 0 0 0 0 0 0 0 0 0 
2 0 0 0 -l -1 -1 0 0 0 0 0 0 0 
+1 0 0 +1 0 -1 -1 -1 -1 +1 0 0 0 
0 +1 0 0 +1 0 +1 0 0 -1 -1 -1 -1 
5 0 0 0 0 0 0 0 +1 0 0 +1 0 0 
0 0 0 0 0 0 0 0 +1 0 0 0 +1 0 
7 0 0 0 0 0 0 0 0 0 +1 0 0 0 +1 
0 0 +1 0 0 +1 0 0 0 0 0 0 0 (0) 


b) Associating nodes 1 to 4 with the 4 rows of the matrix, we construct the digraph 
oy inserting an arc for each column of the matrix. The arc leaves the node of the row 
where the column has a —1 and enters the node of the row where it has a +1. 


| 0.2 CYCLE DIRECTIONS FOR NETWORK FLOW SEARCH 


Linear programming algorithms of Chapters 5 and 6 center on constructing improving 
feasible directions, directions of solution change that preserve feasibility and improve 
the objective function for suitably small steps. Knowing that network flow problems 
are linear programs with special properties, it should not surprise that unusually 
simple improving feasible directions are at the heart of their tractability. 


Chains, Paths, Cycles, and Dicycles 


Sections 9.1 introduced the notions of paths and dicycles in a graph. To derive 
directions for network flow problems, we need the additional concepts of chains and 
cycles. 


10.8 | A cycle is a chain with the sa e beginning and ending node. 


We describe a chain or cycle merely by listing its arcs in sequence. When no confusion 
will result, we may also use the corresponding sequence of nodes. 
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Figure 10.4(a) illustrates on the OOI digraph of Figure 10.2. The first example 
shows a chain (1, 3), (3, 6), (4, 6), (2, 4) connecting nodes 1 and 2. It could just as 
well be called by its node sequence 1-3-6-4—2 because there is only one arc that 
could provide each of the implied node-to-node connections. Figure 10.4(a)’s second 
example is chain (1, 3), (3, 4), (4, 7). Here the node sequence would not be definitive 
because both this chain and (1, 3), (4, 3), (4, 7) have node sequence 1-3—4-7. 

Part (b) of Figure 10.4 shows some sequences that are not chains. The first is 
not connected, and the second repeats node 3. 

Notice that chains need not observe direction on the arcs. This is how a chain 
differs from a path. 


10.9 | Paths are chains that transmit all arcs in the forward direction. 


Thus highlighted sequence 1-3-6—4—2 in Figure 10.4(a) is a chain but not a path; it 
violates direction on arcs (4, 6) and (2, 4). Second sequence 1-3-4~7 is both a chain 
and a path. 

The only additional element with cycles is beginning and ending at the same 
node. For example, Figure 10.4(c) includes cycle 1-3-6-4-1 starting and ending at 
node 1 and cycle (3, 4), (4,3) beginning and ending at node 3. Part (d) of the figure 
confirms that disconnected arc sequences or ones repeating a node cannot constitute 
cycles. 

As with chains and paths, the distinction between cycles and dicycles involves 
direction. 


10.10 | Dicycles are cycles that have all arcs oriented in the same direction. 


Thus the first cycle 1-3-6—4-1 of Figure 10.4(c) is not a dicycle because it violates 
direction on arcs (4, 6) and (1, 4). Second cycle (3, 4), (4, 3) does meet the definition 
of a dicycle because it passes both arcs in the forward direction. 


Cycle Directions 


Cycles can pass arcs in either a forward (with direction) or the reverse (against 
direction) manner. Cycle directions derive from this visitation pattern. 


10.11 | A eyele direction of a minimum cost ne 
forward arcs and decreases flow on reverse ai 


twork flow model increases flow on 
of a cycle in the given digr: 


For example, consider the first cycle of Figure 10.4(c): 1-3-6—4-1. With the first 
two arcs forward in the cycle and the last two reverse, we obtain the cycle direction 
sketched below. 
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forward 


Forward arcs in the cycle have +1 components, reverse arcs have —1, and all other 
arcs have 0). ; 

Notice that it matters which way we orient the cycle. If we used the same 
cycle as above but thought of visiting it in sequence 1-4-6-3-1, arcs (1, 4) and (4, 6) 
would be forward arcs with Ax = +1, and arcs (3, 6) and (1, 3) would be reverse with 
Ax = —1. 


Maintaining Flow Balance with Cycle Directions 


The main, flow conservation constraints of minimum cost network flow problems 
in matrix format (10.4) are equality constraints Ax = b where A is the node-arc 
incidence matrix of the given digraph and b is the net demand vector. We know 
from our earliest investigation of improving search (principle [3.29], Section 3.3) 
that any direction Ax preserving feasibility in such equality constraints must satisfy 
the condition AAx=0. We are interested in cycle directions because they satisfy this 
net-change-zero condition for node-arc incidence matrices A. 


10.12 | Adjusting a feasible flow along a cycle direction of a network flow model 
__ leaves flow balance constraints satisfied. _ - — 
To see that this is true, recall that each node of the cycle is touched by exactly two 
arcs, and think about the four ways two arcs can visit a node. Figure 10.5 illustrates 


how node-arc incidence matrix signs of the direction combine with those of the 


Ax Ax 
S Ve +1 ® a ® -1 ®) 
A +1 -1 : < +1 +1 
(a) Forward-forward (b) Forward-reverse 
Ax—> Ax — 
ee" -e e_e-"-¢ 
7 eal -1 Fis -1 “+1 
(c) Reverse-forward (d) Reverse-reverse 


FIGURE 10.5 Possible Ways a Cycle Can Visit a Node 


10.2 Cycle Directions for Network Flow Search 


direction | 10,11 | to produce a net change = 0 at each node. In the forward—reverse 
case, for example, we have 


+1 Axij + 1 AxKj = +1(4+1) + 1(-1) =0 


SAMPLE EXERCISE 10.5: CONSTRUCTIN 


Consider the following digraph: 


Construct the cycle direction for each of the following cycles, and verify that each 
retains flow balance at node 6. 


(a) 1-2-7-6-5-1 
(b) 3-4-6-3 
Analysis: 
(a) Applying definition [10.11], the cycle direction has 
AX1.2 = Axs; =+1 
Ax7,2 = Ax67 = Axs6=—1 
Ax13 = Ax34 = Ax42 = Ax6.3 = Axa6 =0 


Checking principle | 10.12 | at node 6, 


—1Ax67+1Axs5.6 = —1(-1) + 1(-i) =0 


(b) Again applying definition | 10.11), the cycle direction has 


AXx3,4 = AX4,6 = AX6,3 =I 


Axi. = AX13 = Axo4 = Ax7 2 = Ax6.7 = Axs6 = Axs1 = 0 
At node 6 


+1 Axg6 —1Ax63 = +141) -—1441) =0 


Feasible Cycle Directions 

Flow balance equalities are not the only constraints of a minimum cost network flow 
model [703 |. We must also be concerned with nonnegativity constraints and arc 
capacities u;;. Section 3.3 principles | 3.27 | and | 3.28 | tell us the requirements that 
must be added for upper and lower bounds. No arc with zero flow can decrease, and 


no capacitated arc can increase if bound constraints are also to be maintained in a 
move from a current feasible solution x. 
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current solution x if and 
< u;; on all forward arcs. 


To illustrate, we need a feasible flow. Consider the OOI flow x depicted in 
Figure 10.6. That x satisfies net demand requirements at all nodes and conforms 
to all bounds. 


FiGurE 10.6 Initial Flow x in OOI Example 


One cycle direction corresponds to arc sequence (2, 3), (3, 4), (2, 4): 


We increase flow on forward arcs (2, 3) and (3, 4), and decrease on reverse arc (2, 4). 
Principle | 10.12 | assures that a move in this cycle direction will preserve flow balance 
at all nodes. Also, both forward arcs have infinite capacity, so that an increase will 
violate no upper bound. Still, reverse arc (2,4) has current flow zero. It cannot 
feasibly decrease, so this cycle direction is not a feasible direction. 


Figure 10.7 shows a cycle 2-4-7—3-2 that does satisfy conditions| 10.13). Both 
reverse arcs have cae > 0, and neither forward arc has a capacity. 
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FIGURE 10.7 An Improving 
Feasible Cycle Direction at 
x) of Figure 10.6 


cost changes. +S 
SAMPLE EXERCISE 10.6; IDENTIFYING FEASIBLE CYCLE DIRECTIONS 


The figure that follows shows a network flow problem with a feasible flow x on 
highlighted arcs. Labels on arcs indicate capacities and flows (uj,;, x;,). 


(200, 125) (300, 125) 


Determine whether the direction for each of the following cycles is a feasible cycle 
direction. 

a) 1-2-7-6-5-1 

b) 3-4-6-3 

c) 1-3-6-5-1 


Analysis: We apply conditions| 10.13}. 


a) This cycle direction is feasible because all reverse arcs currently have positive 
flow and no forward arc is at capacity. 


5) This cycle direction is not feasible. Forward arc (3, 4) is at capacity and cannot 
increase. 


c) This cycle direction is not feasible. One of the reverse arcs, (3, 6), has current 
Sow 0 and cannot decrease. 
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Improving Cycle Directions 

Feasible cycle directions aid a search for an optimal flow only if they improve the 
objective function. We know from principle (Section 3.3) that this will be true 
if¢ 4e- Ax <0 for our minimizing model. The simple +1, —1, 0 coefficient structure 
of cycle directions (principle [10.11 ]) make this test particularly easy to apply. 


cAc-Ax 
— 3 Cij AXjj 
(ij) in cycle 
= Ss cj(+)) + Ss ci(=1) 
forward (i,j) reverse (i,j) 


= (total forward arc cost) — (total reverse arc cost) 


10.14 af A cycle direction i 1 improves fora minimum cost network flow model if 


Cycle 2-4-7-3-2 of Figure 10.7 illustrates a direction that is both feasible and 
improving. Applying | 10.14 | to check the latter gives 
(total forward) — (total reverse) = (7+ 5) — 17+ 4) = -9 <0 


SAMPLE EXERCISE 10.7: IDENTIFYING IMPROVING CYCLE DIRECTIONS 


The figure that follows adds costs to the network of Sample Exercise 10.6, Labels on 
arcs now show costs, capacities, and current flows (cj, Wij, Xij). 


(15, °°, 0) 


(23, 200, 125) (7, 300, 125) 


Determine whether each of the following cycles yields an improving cycle direction. 
(a) 1-2-7-6-5-1 

(b) 2-4-6-7-2 

(c) 6-3-1246 
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Analysis: We apply condition | 10.14 |. 
(a) This cycle direction improves because 

(total forward) — (total reverse) = (8 + 15) — (294.74 23) = — 
(b) This cycle direction does not improve because 

(total forward) — (total reverse) = (10 — 20 + 7 + 29) — (0) = 26 
(c) This cycle direction improves because 


(total forward) — (total reverse) = (0+ 8 + 10 — 20) — (20) = — 


Step Size with Cycle Directions 


If we can take an arbitrary step A in improving feasible direction Ax, we know that 
our model is unbounded. The objective can be improved forever without losing 
feasibility. 

More often, upper and lower bounds on arc flows impose limits. Each unit step 
in a cycle direction increases forward arc flows by a unit and decreases reverse flows 
by aunit. Feasibility will be lost the first time these changes encounter a nonnegativity 
or capacity constraint. More specifically, 


10.15 l Steps from feasible flow x in cycle direction Ax retain feasibility 10k ae 
sizes toe aun} 2 where = 


Return to the cycle direction 2-4-7-3—7 of Figure 10.7 and the flow x of Figure 
10.6. For that example, 


At = min{(co — 0), (co — 0)} = 

A~ = min{610, 1000} = 610 
Thus A = min{oo, 610} = 610 is the largest step we can take in that cycle direction 
without losing feasibility. 


E DIRECTIONS 


eran t to the Care of ee Exercise 10.7. Whether or not directions corre- 
sponding to the three specified cycles improve the objective function, determine the 
maximum step A in each direction that preserves feasibility. 


Analysis: We apply principle | 10.15. 
a) For cycle 1-2-7-6-5-1, 

at = min{(100 — 0), (co — 0)} = 100 

Kae — min {125012539125}. = 125 

X = min{at, a7} = min{100, 125} = 100 
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(b) For cycle 2-4-6-7-2, 
AT = min{(oo — 0), (90 — 0), (300 — 125), (co — 125)} = 90 
A” = +00 
A = min{At, A7} = min{90, co} = 90 
(c) For cycle 6-3-1-2-4-6, 
AT = min{(oco — 0), (100 — 0), (00 — 0), (90 — 0)} = 90 
A~ = min{35} = 35 
h -= minfA*, A7}—-min{90,.35} = 35 


Sufficiency of Cycle Directions 


Properties we long ago encountered for linear programs of any form, including net- 
work flow models of present interest, make a solution globally optimal if and only if 


it admits no improving feasible directions (principle | 5.1 }). We have seen how cycle 


directions can be improving and feasible. But what if none exist that satisfy both 
requirements? Certainly there are more complicated improving feasible directions 
for network flow models. Could any improve a solution when cycle directions fail? 
Happily, no. 


ible flow in a minimum cost network flow problem i 
dmits no improving feasible cycle direction. 


A formal proof would be beyond the scope of this book, but an example will 
illustrate why | 10.16 | must hold. The key insight is a decomposition: 


ee is feasible and improving, at least 
feasible and i improving. 


2 


Figure 10.8 A Non-Cycle Improving 
D Feasible Direction at x©) of Figure 10.6 
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Consider the direction shown in Figure 10.8. It certainly does not look like a 
cycle direction, but the reader can check that it satisfies conditions to be both feasible 
and improving. The objective changes by ¢ = —$15.9 for every unit step. 

This complex direction decomposes into a sum of the cycle directions for cycles 
3-4-7-3, 1-3-2-8-1, and 2-4-7--3-2 as follows: 
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The three component cycle directions have reduced costs ¢ = —12.0, +3.0. 
and —1.8, respectively. We would never need the more complex direction because 
both the first and the third cycle directions are improving and feasible at x of Fig- 
ure 10.6. 


19,3 RUDIMENTARY CYCLE DIRECTION SEARCH 
ALGORITHMS FOR NETWORK FLOWS 


Section 10.2 has developed all the building blocks. We are now ready to specify cycle 
direction-based algorithms for minimum cost network flow models. 

Most of the known procedures for minimum cost network flow problems can be 
viewed as special forms of improving search using cycle directions. Every iteration 
identifies an improving feasible cycle direction and adjusts flows along it to reduce 
cost without losing feasibility. If no such cycle direction exists, the present solution 
is optimal (principle | 10.161). 

Algorithm 10A provides a formal statement, but we omit for the moment many 
important details. In Sections 10.7 and 10.8 we specify schemes for determining at 
Step 1 whether any improving feasible cycle directions exist and for constructing 
them as needed at Step 2. For the moment these steps will be done by inspection. 


Rudimentary Cycle Direction Search of the OOI Example 


Figure 10.9 summarizes costs and capacities of the OOI example, along with the 
starting feasible flows of Figure 10.6, which have cost $32,540. Figure 10.10 then 
details an application of Algorithm 10A to compute an optimal flow. 

We first apply the improving feasible cycle direction of Figure 10.7. For that 
cycle A* = +00 and A~ = min{610, 1000} = 610. A maximum feasible step of 
i = 610 in that direction yields flow x at cost $27,050. 


(c, u, x) ae 


(7, 2°, 560) 


1000 


(0, ce, 440) 


440 ¢ (6) soo 


(0, ©, 0) 


-1000 


FiGurE 10.9 Data and Initial Flow for the OOI 
Example 


| 
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Step 0: Initialization. Choose any starting feasible flow x, and set solution 
index t — 0. 
Step 1: Optimal. If no improving feasible cycle direction A x exists at current 


solution x® (conditions| 10.13 |and| 10.14), then stop. Flow x is globally 
optimal. 

Step 2: Cycle Direction. Choose an improving feasible cycle direction Ax 
at x. 

Step 3: Step Size. Compute the maximum feasible step % in direction Ax 


(rule i073 )): 


At <& min{(ui; —x()): (i, j) forward} (too if none) 


@. 


AT <— min{x;; : Gj) reverse} (+00 if none) 


A <— min{At, a7} 


If 1. = 00, stop; the model is unbounded. 
Step 4: Advance. Update 


xD — x94 Ax 


by increasing flows on forward arcs of the cycle direction and decreasing 
those on reverse arcs by the amount A. Then increment t <— t+1 and return 
to Step 1. 


Next we employ cycle 2-3-1—-4-2 at ¢ = —2 and 4 = 560 to produce flow x” at 
cost $25,930. A final step in the direction for the indicated cycle 2-3-4-2 completes 
recovery of the optimal flow in Figure 10.3 with value $25,855. 

The reader can verify that each of the three directions employed was both 
improving and feasible for the solution to which it was applied. However, many 
other choices were available. Pending the development of Sections 10.7 and 10.8, 
readers should assume that choices were made arbitrarily, with appropriate cycles 
discovered by trial and error. 

Many cycle directions remain in the optimal flow x®. Some are either improv- 
ing or feasible. Still, it can be shown that no cycle direction is both improving and 
feasible. Algorithm 10A terminates with a conclusion of optimality. 


SAMPLE EXERCISE 10.9: APPLYING RUDIMENTARY ALGORI HM 104 


Consider the network flow problem depicted below. Labels on nodes iedionte net 
demands bz. Labels on arcs show cost, capacities, and current flows (cj;, uj;, Xij)- 
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t Flow x Cycle Direction Ax Step 
0 
1 
2 
No improving feasible cycle 
3 directions exist. Flow x’ is 
optimal. 


FIGURE 10.10 Rudimentary Cycle Direction Solution of OOI Example 


es OT UR MS Me — 
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Verify that the direction for cycle 1-2—4-1 is improving and feasible. Then take the 
maximum feasible step in this direction and show the resulting improved flow. 


Analysis: The direction is feasible under principle | 10.13 | because reverse arc (1, 4) 
has positive flow, and neither forward arc (1, 2) nor (2, 4) is at capacity. The direction 


é=(12+5)— G6) =—-19 <0 
Application of step size rule | 10.15 | gives 
A* = min{(35 — 20), (100 — 0)} = 15 
A~ = min{30} = 30 
A = min{15, 30} = 15 


Thus flows increase by A = 15 units on forward arcs (1,2) and (2, 4) of the cycle, 
while decreasing by 15 units on reverse arc (1, 4). The result is 


Finding Starting Feasible Solutions 


As with all constrained optimization models, Algorithm 10A can be adapted to find 
a starting feasible flow or establish that none exists. We may employ either the 
*wo-phase or the big-M scheme of Section 3.6. 

The key to both those methods is construction of an artificial model from which 
computation can begin. New artificial variables are introduced to force feasibility. 
and their sum is minimized. If that sum can be driven to = 0, what remains is a 
feasible flow for the original model. If not, the original model is infeasible. 
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Artificial Network Flow Model 


The only new element in the network flow context is that we wish to create an artifical 
model that is itself a minimum cost network flow problem so that Algorithm 10A 
applies. In particular, we want to be able to interpret artificial variables as flows in 
arcs. 

To obtain such a network flow artificial model, we simply put a zero flow on all 
arcs of the original model and add one artificial node. Supply and demand require- 
ments are fulfilled by artificial arcs joining this special node to all others having net 
demand b; 4 0. 


required demand |b,|. 


Figure 10.11 illustrates for our OOI model. Artificial node 0 has been added 
to anchor artificial arcs. Then artificial arcs (1, 0) and (2, 0) balance flow at supply 
nodes 1 and 2 by carrying exactly the supply specified at those nodes. Similarly, 
artificial arcs (0, 5), (0, 6), (0, 7), and (0, 8) satisfy demand requirements by bringing 
the needed flow to each demand node. All original arcs have flow = 0. 


FicurE 10.11 Starting Phase I Flow for the OOI Example 


Zero flows on all original arcs satisfy their upper- and lower-bound constraints, 
and allspecified artificial flows are nonnegative. Also, setting artificial flows to exactly 
the needed supply or demand has assured flow balance at all the original nodes. 


aa oh aS. 
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But what about flow balance at the new, artificial node? To see that it too is 
balanced, observe that inflow will sum to total supply, and outflow will sum to total 
demand. Under total supply equals total demand requirement 10.5], these totals 
may be assumed to agree. 


SAMPLE EXERCISE 10.10: CONSTRUCTIN 


, ARTIFICIAL NETWORK MODEL 


Consider a network flow problem with net demands as depicted in the following 
igure: 


Construct both the corresponding artifical network model for phase I or big-M com- 
putation of a starting feasible solution and the associated artificial solution. 


Modeling: Following construction | 10.18), we introduce an artificial node 0, along 
with artificial arcs from supply node 2 and to demand nodes 3 and 4. Flows on original 


arcs are all zero, and those on artificials equal the specified supply or demand. The 
result is the following artificial model and artificial starting solution: 


Network Flow Algorithms versus Linear Programming 


Every network flow model is a linear program, so we could solve any of them with 
the simplex or interior point methods of Chapters 5 and 6. Still, implementations of 
cycle direction Algorithm 10A are usually faster. 


efficient. 


To see why, glance back at algorithm statements of Chapter 5 and 6. Even 
the most efficient revised simplex version 5C in Section 5.8 had to solve linear 
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systems of equations to compute pricing vector (dual solution) v and simplex di- 
rection Ax. Interior point algorithms all require relatively complex projection com- 
putations (Sections 6.1 and 6.2). 

Assuming that we can find improving feasible cycle directions without too much 
effort, which we will show how to do in Sections 10.7 and 10.8, the network com- 
putations of Algorithm 10A are trivial by comparison. We have only to develop a 
labeling scheme to mark the improving feasible cycle, pass around it to determine 
step size 4, and pass around again to add or subtract 4 on forward and reverse arcs, 
respectively. Not one multiplication or division is required. 


10.4 INTEGRALITY OF OPTIMAL NETWORK FLOWS 


As far back as Chapter 2, we have seen that integer linear programs (ILPs) are 
generally much less tractable than linear programs (LPs). We are now prepared to 
show why network flows are often an exception. Under mild conditions, optimal 
network flows automatically take on integer (whole-number) values. Thus, if an 
ILP can be shown to have network form, we may solve it with methods based on 
Algorithm 10A which are even more efficient that general-purpose LP methods of 


Chapters 5 and 6 (principle | 10.19 }). 


When Optimal Network Flows Must Be Integer 


Suppose that all constraint data of a given minimum cost network flow model— 
supplies, demands, and capacities—happen to have integer values as they do in the 
OOlexample of Figure 10.9. Then we can think through some observations about the 
steps in a two-phase or big-M implementation of Algorithm 10A that have important 
consequences. 


eThe starting artificial flow of construction | 10,18] will be integer because the 
only nonzero flows occur on artificial arcs, and all those flows equal supplies or 
demands that we have assumed integer. 


eRegardless of whether we proceed by two-phase or big-M application of Al- 
gorithm 10A, an optimal solution can be computed using only cycle directions 


(principle )s 


eEach iteration of Algorithm 10A either increases arc flows by step size A, de- 
creases them by A, or leaves the flows unchanged, because all components of 


cycle directions are +1, —1, or 0 (construction | 10.11 |). 


eWhenever the current flow x is integer, step size 1 of | 10.15 |, which will equal 


iG) 


: t t), : : 
either some x; j or some (ij — x P ), must also be integer under our assumption 


of integer u;;. 


The consequence is a fundamental integrality property. When a network flow 
model has integer constraint data, we can start with an integer flow and execute every 
step of two-phase of big-M solution by adding or subtracting integer quantities ) to 
current flows. If the result is an optimal solution (i.e., not infeasible or unbounded), 
that solution must have integer flows on all arcs. 
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r constraint data (sup-_ 
, it has an integer opti- 


mal solution, | 


Notice one assumption not required in principle | 10.20). Nothing was said 


about arc costs c;;. Integrality property depends only on constraint data. 
What is important is that flows start integer and change by integer amounts at each 
step. Costs have no effect because only supplies, demands, and capacities are involved 
in flow adjustments. 


Each of the following details the constant dats fia a minimum cost intebirark flow 
problem. Assuming that the models are neither infeasible nor unbounded, determine 
whether each must have an integer optimal solution. 


a) b = (100, 200, 0, —300), u = (90, 20, 00, 220, 180), ¢ = (8, 9, —4, 0, 6) 

b) b = (—30, 40, —10, 0), u = (20, 123, 23, 15, 92), e = (11, 0, 3, 81, 6) 

c) b = (-134, —20, 234, 10), u = (10, 20, 00, 0, 40), e = (—4, 8, 0, 19, 31) 

d) b = (25, 15, 0, —40), u = (20, 00, 30, 45, 10), e = (3.5, 9.6, —2.1, 11.77, V2) 


Analysis: We apply principle | 10.20). 


a) This model has integer supplies, demands, and capacities. An integer optimal 
solution will exist. 


5) This model has a fractional capacity. An integer optimum is not assured. 
c) This model has a fractional supply and a fractional demand. An integer optimum 
s not assured. 


a) Even though cost data are highly noninteger, this model has integer constraint 
data. An integer optimal solution will exist. 


Total Unimodularity of Node-Arc Incidence Matrices 


the above “starts integer and stays integer” arguement is only one way of seeing 
‘hat network flow optima will be integer when all constraint data are integer. Since 
»e know that optimal solutions, at least unique ones, must be extreme points of the 


corresponding LP-feasible region (Section 5.1, principle ), there must also be 
something special about the basic solution computations of extreme points (Section 
5.2) in minimum cost network flow models. 

The property is called total unimodularity: 
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The main constraint matrices of network flow models are their node—are incidence 
matrices, and they have this very special property: 


Ree Node-are incidence! matrices ’ network = models are ay unimod- 
: ular. . EEE 2 : . = : 


The relevance of total unimodularity comes in solving for basic (i.e., extreme- 
point) solutions to network LPs. The famous Cramer rule for solving systems of linear 
equations shows that the maximum denominator of the result is the determinant of 
the corresponding basis matrix. Under total unimodularity, that denominator will 
always be +1, meaning that no fractions will be introduced. 


It is beyond the scope of this book to establish principle | 10.22 |in general, but 


an example from the OOI node-arc matrix of Table 10.1 will illustrate. Extracting 
rows for nodes 2, 3, and 4, along with columns for arcs (2, 3), (2, 4), and (3, 4), gives 


(2,3) (2,4) (3,4) 


2\/ -1 -1 0 
3} +1 Ol 
4 QO -1 +1 


All 1 by 1 submatrices are obviously +1, 0, or —1. The full 3 by 3 matrix has deter- 
minant = 0, its lower left corner yields 


+1 0 
act ( mn jes 


and all other 2 by 2 submatrices are similar. 


10. 5 TRANSPORTATION AND ASSIGNMENT MODELS 


The OOI example of Section 10.1 contains all the major elements of network flow 
models, but it only begins to suggest the rich variety of problems that can be treated 
as networks. In this section we present the classic transportation and assignment 
special cases, and in Section 10.6 we introduce several more. 


Transportation Problems 


Among the simplest of classic flow models are transportation problems. 


for which every nade 1S ecitier a pure supply node Cm arc points out) or a 
~ pure demand node (every ¢ are points i in). 


That is, all flow goes immediately from some source node, where it is supplied, to a 
sink node where it is demanded. There are no intermediate steps and no transship- 
ment nodes. The commodity flowing in a transportation problem may be people. 
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»ater, oil, money, or almost anything else. It is only necesary that flows go direct 
‘rom sources to sinks. 


Each of the following digraphs shows a minimum cost network flow problem. Num- 
>ers on vertices are net demands, b;, and those on arcs are costs. 


(b) 
Determine which are transportation problems. 


Modeling: 


a) This network flow problem is not a transportation problem because node 3 is a 
sure transshipment node, with neither supply nor demand. 


>) This network flow problem is also not a transportation problem because nodes 2 
and 3 have both inbound and outbound arcs. 


©) This network flow problem is a transportation problem. Nodes 1 and 3 are pure 
source nodes; nodes 2 and 4 are pure demand nodes. 


Standard Form for Transportation Problems 
Using constants 
s; supply at node i 
d; & demand at node j 


cij & unit cost of flow from i to j 
end decision variables 


xj, & flow from i to j 
the network flow formulation of a transportation problem simplifies to 
min DD cutis 
I 
st. - Ds Kij = Si for alli 
i 
Dept ed for all j 


xij; = 0 for all 7,7 
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As usual, flows leaving supply nodes carry negative signs, and those entering demand 
points have positive signs. 

The more common statement of transportation problems in operations research 
uses the standard form obtained when signs are reversed on the first set of constraints: 


10,24 L For supplies s;, ;.demands dj, and eO51Sc the standard form of aes 
tion problems is. = 


Sa 


70 ies 
a x = dj Zi se ally 
x; = 0 Tot all i,7 


SAMPLE EXERCISE 10.13: FORMULATING TRANSPORTATION PROBLEMS 


Write a standard-form transportation problem formulation for the diagraph of Sam- 
ple Exercise 10.12(c). 


Modeling: Following | 10.24 |, the model in standard form is 


min 7x12 + 5x1,4 + 4x32 + 9x34 


Sl. X12 ea = 50 
X31 + 3,4 = 30 
Kio vee = 45 
X14 + X34 =) 3)5) 


H95,X1,45.X3\1, X34 0 


EXAMPLE 10.2: MARINE MOBILIZATION TRANSPORTATION 
PROBLEM 


A really massive transportation problem example arises in officer mobilization plan- 
ning for the U.S. Marine Corps.' During any international emergency, thousands of 
officers must be mobilized from their regular duty or reserve positions into billets 
required for the emergency. However, not every officer is qualified by rank, train- 
ing, or experience to fulfill every assignment. Using the very efficient network flow 
methods of this chapter, Marine planners are able to develop a mobilization scheme 
in a few minutes by solving a transportation problem with over 100,000 arcs. 
Mobilization options can be depicted in a digraph like the fictitious one of 
Figure 10.12. Supply nodes exist for each group of like-qualified officers presently 
based in the same location. For example, Figure 10.12 shows one supply node for 
captains now in the Ist Division who are trained as intelligence officers. Another 
represents civil affairstrained officers in Georgia units of the Marine Corps Reserve. 


TBased on D. O. Bausch, G. G. Brown, D. R. Hundley, S. H. Rapp, and R. E. Rosenthal (1991). 
“Mobilizing Marine Corps Officers,” Interfaces, 21:4, 26-38. 
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AVAILABLE NEEDED 


Headquarters 


Captain, 1st Division 
Tactical Intelligence 


Intelligence 


Local Government 


Captain, 1st Division : 
Liason 


Field Artillery 


Force 1 


Major, 2nd Division : 
Embarkation 


Embarkation 


Force 1 


Captain, Georgia Reserves pats 
Communications 


Civil Affairs 


normal arcs > 


high-cost artificial arcs 


artificial for unfillable billets 


FiGuRE 10.12 Marine Mobilization Transportation Problem 


Demand nodes indicate needs for officers of particular qualifications at active loca- 
tions of the emergency. For example, Figure 10.12 depicts the need for one or more 
local government liason officers in the forward area of a deployment. 

Arcs exist whenever officers represented by a supply node would be qualified 
to fill billets of ademand node. Thus the Georgia civil affairs officers are linked to the 
local government liason billets but not to artillery or intelligence needs. There may 
be several feasible assignments for officers of any source node. For example, Figure 
10.12 indicates that the same civil affairs officers could serve as communications 
officers with assault force 1. 

The Marines’ first priority is to fill all needed billets, but there are always some 
left unfilled. The transportation problem of Figure 10.12 models the possibility of 
unfilled billets with an artificial supply node connected to all demands. High cost on 
its arcs penalizes unfilled billets in the objective function. 

Once as many requirements as possible have be filled, a secondary concern is 
to minimize turbulence. That is, the Marines try to assign officers to the same unit to 
which they were assigned before mobilization, or at least to assign them to a nearby 
unit by minimizing total travel cost. 

Using the following notation: 


s; A supply of officers available at source node i 
d; & demand for officers at demand node j 
ci; & distance officers at supply node i would have to travel to report for billet j 


(a large positive number if i is the artificial node) 
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I; 4 set of officer supply nodes i suitable for billets j 
J; 4 set of billet demand nodes j suitable for officers at supply node i 


x;; 4 number of officers at node i mobilized to billets at j 
this Marine mobilization problem reduces to the standard-form transportation 


problem 
min a Se Ci jXij 
ae 


Se Xj = 8; for alli (supply of 7) 
jedi 

ye Ki = Oh for all j (demand for /) 
iel; 


My 0) for all 7, j 


Assignment Problems 


Another important class of network flow models, known as (linear) assignment prob- 
lems, do not seem at first glance to have anything to do with something flowing. 


10.25 | Assignment problems deal with optimal pairing or matching of ne 
in two distinct sets. 


We might pair jobs to machines, male dating service clients to female, duties to 
employees, and so on. 

Assignment problems are modeled using (discrete) decision variables 
1 if / is assigned to j 
Xij i 3 
0 otherwise 
The first subscript refers to items in one set; the second subscript identifies items in 
the other. 

Assignment problems may have quite complicated objective functions (see 
Section 11.4). We address here only the most common form with a linear objective. 
In that case we have known costs 


ci; 4 cost of assigning i to j 
An optimal solution minimizes (or maximizes) total cost (or benefit). 


Denoting by A the set of allowed assignments (i,j), we can express the linear 
assignment model as 


min or max » Gia a (min or max total cost) 
(ij)eA 
s.t. Des Veal for alli (every i is assigned) 
jwith @,eA (10.5 
Ss Lip —oll for all j (every j is assigned) 
iwith (ied 
xij =Oor1 forall @j)<¢A 
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The first system of constraints guarantees every / is assigned exactly once by summing 
over all possible assignments j. The second system does the same to assure that every 
jis assigned exactly once. 

Notice that this formulation is an integer linear program (ILP). Decision vari- 
ables are allowed only the discrete values 0 and 1. Still, we will soon see how the 
discreteness can be ignored in computing an optimum. 


The following table sows a ARS roti 5 service’s  ompatibiy ratings for 


male customer i = 1,..., 3 with and female customers j = 1,...,3. 
Se fe eae 
i 1 2) 8} 
TP 90) 430 12 
De 4A0r 80) 75 
3 sO MaOa» 2 a80) 


Formulate an assignment model to find the highest compatibility arrangement pro- 
viding each customer with a single date of the opposite gender. 


Modeling: Using variables x;; = 1 if male i is paired with female j, the model corre- 
sponding to system (10.5) is 


max 90x11 + 30x1,2 + 12x13 + 40x2,1 + 80x2,2 + 75x2,3 + 60x31 + 65x3,2 + 80x3,3 


s.t. X41 +412 +413 = 1 
X21 + 2,2 +%23 =1 
X31 +432 +x33 =1 
X11 +21 +431 =1 
X12 + X2,2 + x32 =1 
X13 + X23 + x33 = 1 
all Xij = Oorl 


EXAMPLE 10.3: CAM ASSIGNMENT 


For a more realistic assignment problem, consider a computer-aided manufacturing 
CAM) system that automatically routes jobs through workstations of a computer- 
controlled factory. Each job consists of a sequence of required machining and as- 
sembly operations. 
Often, there are several different workstations where the same operation can 
be performed. Thus the computer control system must make routing decisions. Each 
time that a job completes some operation, the system must select the next station to 
which the job should be routed from among the several that could perform the next 
operation required. 
One method for accomplishing such control decisions in an approximately op- 
timal way is to periodically solve an assignment model. ? To illustrate, suppose that 
the 8 fictitious jobs i of Table 10.2 are either waiting for movement to their next 


2Based on J. Chandra and J. Talavage (1991), “Optimization-Based Opportunistic Part Dispatching 
in Flexible Manufacturing Systems,” School of Industrial Engineering, Purdue University, July. 
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TABLE 10.2 Transportation and Processing Times for CAM 
Assignment Example 


Next Workstations, j 

Jobs,i 1 2 3 4 5 6 v7, 8 9 10 
1 8 — 23 5 — 
2 — 4 — J2) 15) 
3 — — 20 _— 3 6 — 8 —_— —_— 
4 19 10 
5) — _ 8 — —_— 12 — —_— 16 
6 14 8 _ 3 — 
Th — 6 27 — 12 
8 _ 3) 15 32 — —_ 


workstation or will finish their current operation within the next 5 minutes. The 10 
workstations j to which they might be routed are also shown. Entries in the table 
reflect 


transportation waiting time operation 
time to the +] until the station | +] processing time 
station becomes free at the station 


That is, they show the short-term time implications of assigning jobs i to stations j. 
Missing values in the table reflect assignments that are not possible because the next 
required operation cannot be performed at a workstation. 


Balancing Unequal Sets with Dummy Elements 


Letting A 4 {feasible (i, j) pairs}, and c;; 4 the times shown in Table 10.2, an optimal 
short-term control decision is to assign jobs to workstations in a manner minimizing 
total time. This is exactly what assignment model (10.5) will compute. 

One small complication arises from the fact that there are more workstations 
than jobs. Model (10.6) assumes that the sets to be matched have equal numbers of 
objects. 

This problem is easily solved with dummy members. 


10.26. If the two sets to be paired in an assignment problem differ in size, the 
smaller can be augmented with dummy members. These dummy objects should 
be treated as assignable to all members of the other set at zero cost. 


Principle | 10.26 | leads to dummy jobs i = 9, 10 in our CAM example. 


Linear Assignment as Network Flows with Integer Solutions 
Linear assignment model (10.6) is a discrete optimization problem—an ILP. Still, a 


In fact, they are the same. Thinking of i’s as sources with supplies of 1, and 
j’s as sinks with demands of 1, assignment is nothing more than finding an optimal 
source-to-sink transportation flow. 
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What about 0-1 integrality? Network flow integrality property of Sec- 
tion 10.4 completes the transformation. With supplies and demands all = 1 and 
arc capacities also = 1 (after relaxing to 0 < x;; < 1) property | 10.20] assures that 
»ptimal solutions to the underlying network flow linear program will be integer val- 
ued. That is, the network structure of the assignment problem yields integer optimal 
solutions from a linear program. 

In fact, we can ignore the capacities on arcs as well because each x;; is con- 
strained by a source constraint }°;x;; = 1. The result is the standard form for 
assignment models. 


10.27 | Every linear assignment problem can be formulated in standard form and 
solved as the network flow (or transportation problem) linear program 

: min or max - a CipXij : = 

st. yt = 1 for all 

Spe foray 

oe for all i and j 


where ¢;, is the cost or benefit of assigning i to j, and x;; = 1 ifZis assigned to j 
and = 0 otherwise. 


In fact, assignment algorithms are known that are even more efficient that general 
network flow procedures (see, e.g., the Hungarian algorithm in references at the end 
of this chapter). 


SAMPLE EXERCISE 10.15; FORMULATING ASSIGNMENT PROBLEMS AS NETWORK 
: FLOWS — 


Formulate the assignment problem of Sample Exercise 10.14 as a minimum cost 
network flow problem, and explain why optimal flows must be integer. 


Modeling: We must first reverse the sign of all objective function coefficients to 
convert the maximize problem of Sample Exercise 10.14 to minimize form. Then, 
treating i = 1,2,3 as sources with unit supply, and j = 1,2,3 as sinks with unit 
demand, gives the following network: 


Optimal flows will be integer (binary) because supplies and demands are integer, 


and there are no capacities (property | 10.20 |). 
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CAM Assignment Example Model 


We are now ready to state a full formulation of our CAM example in the standard 


format of : 


min 8x11 + 23%1,3 + 5x19 + 4x2.2 + 12x,4 + 15x25 

+ 20x33 + 13x35 + 6x3.6 + 8x39 + 19x45 + 10X46 

+ 8x54 + 12x57 + 116x510 + 14x61 + 8%6.7 + 3X6,9 

+ 6%7,2 + +27x7,3 + 12x7,10 + 5xg.2 + 15x83 + 32x88 
s.t. 


M11+%134+X19 =1 
X2,2 + X2,4 + X25 =1 
X33 + x35 +%3,6 +X3,8 =1 
X45 + X46 = 
X5,4 +%5,7 +X5,10 il 
X61 + X67 + X6,9 =1 
X7,2 + X7,8 + X7,10 = il 
Xg,2 + X83 +Xg.g =1 
X91 + X92 + X9,3 + X9,4 + X9,5 

+ X96 + X9,7 + X9,g + X9,9 + X9,10 =1 


X10,1 + X10,2 + %10,3 + 10,4 + X10,5 
+ X10,6 + X10,7 + X10,8 + X10,9 + X10,10 = 1 


X1,1 + X61 4+ X91 + X101 =1 
X22 + X7,2 + X82 + X9,2 + X10,2 =1 
X1,3 + X3,3 + X83 + X03 + X10,3 =1 
X24 +%5,4 +X9,4 + X10,4 =1 
2,5 + X35 + X45 +X9,5 + X10,5 =1 
X3,6 +X4,6 + X9,6 + X10,6 =1 
x5,7 + X6,7 + 9,7 + 10,7 ==/il 
X3,8 + X7,3 + X83 + X98 + X10,8 =1 
X1,9 + X6,9 + X9,9 + X10,9 =1 
X5,10 + X7,10 + X9,10 + 10,10 =1 


xij =O or Morallt = 1,-2.5 10;7'= 1, 2, 10 


An optimal short-term routing of jobs to workstations has 


(job 1) 
(job 2) 
(job 3) 
(job 4) 
(job 5) 
(job 6) 
(job 7) 
(job 8) 


(job 9) (10.6 


(job 10) 
(station 1) 
(station 2) 
(station 3) 
(station 4) 
(station 5) 
(station 6) 
(station 7) 
(station 8) 
(station 9) 
(station 10) 


* — * = 2k = 5 * = * — * a, * _ * ot 
X41 = %72 = %3.g = X46 =%X5.4 = X69 =X719 =%33 = 1 


and all other xij =0) 


Tableau Representation of Transportation 
and Assignment Problems 


Especially when all (, j) combinations are feasible, a digraph depiction of a trans- 
portation or assignment problem can become very confusing. That is why such 
problems are often represented in an alternative network tableau format. 


_ en inn Ne ee 


10.28 J "Transportation and assi 
sented in network tableau form 


sinks j, supplies 5; 
demands d; along the bottom, unit costs ¢;; in the st 


To see the idea, return to the transportation problem of Sample Exercise 10.13: 
min 7x1,2 + 5x1,4 + 4%3,2 + 9x34 
st. X12 +x14 = 50 
x31 + x34 = 30 
X12 + X32 = 45 
X14 +2%3,4 = 35 
X1,25 X1,4, X3,1,%3,4 = 0 


with digraph depicted in Sample Exercise 10.12(c). The corresponding tableau would 
have the form 


Nw D 4 
7 5 

1 {5p 6595] 51-90 
4 9 

3 30 \eeee 
dy=45 dy=35 


when 142= 15, X14= 35, X32 = 30, and X34 = 0. 


10.6 OTHER SINGLE-CoMMODITY NETWORK FLOW MODELS __ 


The transportation and assignment models of Section 10.5 are the most famous spe- 
cial cases of network flow problems, but there are many others. In this section we 
develop several more that retain the single-commodity character of all models treated 
so far. That is, the underlying problems can be modeled adequately by simply track- 
ing how much of some commodity enters or leaves the network at each node and how 
much of it flows through each arc. In Section 10.9 we introduce the more difficult 
cases where multiple commodities must be accounted for separately and where flow 
may be gained or lost as it transits the arcs of the network. 
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Network Flow Formulation of Shortest Path Problems 


In Section 9.1 we introduced shortest path problems—models seeking the least cost 
path between specified pairs of nodes in a given graph. For example, the Two Ring 
Circus case reproduced in Figure 10.13 required a shortest path from Lincoln (node 
1) to Tallahassee (node 9). Numbers on arcs reflect travel costs for the circus less 
revenue for appearing in cities along the way. 


ve aad 2) 3 
a 
Springfield 


2 
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FIGURE 10.13 The Two Ring Circus Shortest Path Example 


In Sections 9.3 to 9.7 we detailed very efficient algorithms for shortest path 
problems with no negative-length dicycles. When codes are available, those methods 
provide the most effective solution techniques. 

We can, however, compute shortest paths as minimum cost network flows (and 
thus also as linear programs) by exploiting integrality property 10.20). It is only 


necessary to define appropriate supplies and demands, then solve a flow problem 
over the given shortest path digraph. 


10.29 | Although it is usually more efficient to use dynamic programming-based 
methods of Chapter 9, the problem of finding a shortest path from source s to 
sink 1 in the absence of negative dicycles can be solved as a minimum cost flow 


problem over the same digraph and the same costs, with a supply of 1 at the 
_ source anda demand of | at the sink. All other nodes are transshipment. 


On the Two Ring Circus example of Figure 10.13, construction creates 
a unit supply at Lincoln node s = 1 and a unit demand at Tallahassee sink t = 9. 
Certainly, flow introduced at Lincoln, and conserved at all nodes along the way, will 
eventually reach Tallahassee along one or more s-to-t paths. 

Network integrality property | 10.20) assures that all flow will be along a single 
path because supplies and demands are integer. Those arcs with x;; = 1 will be on 
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the optimal path, and those with x;; = 0 will not be traversed. This single path must 
be a shortest path because no flow following a path longer that the shortest could be 
a minimum cost flow. 


HORTEST PATHS AS NETWORK 


SAMPLE oe 10.16: FORMULATIN 
FLOWS. __ 


source 


Numbers on arcs are lengths c;;. 


a) Formulate a minimum cost network flow model to compute a shortest path from 
node 1 to node 4. 


5) Solve you model by inspection and identify an optimal path. 


Modeling: 


a) Applying construction | 10.29 |, the appropriate model is 
min 3x12 + 15x13 + 2x23 + 9x24 + 4x3 + 6X3,4 


s.t. ei elles) = —!1 
Sei eee) so I a aa 
M3 4. 8 X39) 34 — 
X2,4 + X3,4 = 1 


X1,2+X1,3, X2,3, X2,4, X3,2, 3,4 > 0 


5) An optimal solution to the model of part (a) is x}, = x3, = x34 = 1x73 = 
2 = 3,4 = 0. The corresponding optimal path indicated by x7; = 1 is 1-2-3-4. 


Maximum Flow Problems 
Maximum flow problems are another very simple special case of network flows. 


The only issue is how much commodity can be moved from the single source to a 
single sink. 
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7 


Determine by inspection then maximum flows from neal 1 to node 4 in the following 
graph (numbers on arcs are capacities u;,;): 


SAMPLE EXERCISE 1 


fe. 


source 


Analysis: Careful examination of the possibilities will establish that a maximum flow 
sends 80 units from 1 to 4 as follows: 


source 


EXAMPLE 10.4: BUILDING EVACUATION MAXIMUM FLOW 


Maximum flow problems arise most often as subproblems in more complex oper- 
ations research studies. However, they occur naturally in evaluating the safety of 
proposed building designs.? Proper design requires adequate capacity for building 
evacuation in the event of an emergency. 

Figure 10.14 shows a small example involving a proposed sports arena. Patrons 
in the arena would exit in an emergency through doors on all four sides that can 
accommodate 600 persons per minute. Those doors lead into an outer hallway that 
can move 350 persons per minute in each direction. Egress from the hallway is 
through four firestairs with capacity 400 persons per minute and a tunnel to the 
parking lot accommodating 800 persons per minute. Our interest is in the maximum 
rate of evacuation possible with this design. 

Part (b) of Figure 10.14 shows how we reduce this safety analysis to a maximum 
flow model. Patron flows originate at source node 1. Outbound arcs model the four 
doorways. The flows around the outer hall lead to the four stairways and the tunnel. 
Persons exiting by any of those means pass to sink node 10. Capacities enforce the 
flow rates of the various facilities. 

We wish to know the maximum flow from 1 to 10, subject to the capacities 
indicated. An optimal flow is provided in the arc labels of part (b). Patrons can 
escape at a total rate of 2100 per minute. 


Based in part on L. G. Chalmet, R. L. Frances, and P. B. Saunders (1982), “Network Models for 
Building Evacuation,” Management Science, 28, 86-105. All numerical data and diagrams were made up 
by the author of this book. 
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Ur oo 


(a) Layout (b) Network 


(c) Optimal flow 


FIGURE 10.14 Building Evacuation Maximum Flow Example 


Return Arc Network Flow Formulation of Maximum 

Flow Problems 

As so far presented, neither the tiny example of Sample Exercise 10.17 nor the larger 
oe of Figure 10.14(b) is a minimum cost network flow problem. Flow conservation 


and capacity requirements are much like standard model , but we have specified 
no costs, and flows do not balance at source and sink. 
To create a true minimum cost flow problem, we add a return arc. 


10.31 | Return ares balance unknown s 
flow in an artificial arc from sink to source. 


Adding a return arc to the maximum flow example of Figure 10.14(b) produces 
the following digraph: 


a15 
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Artificial arc (10, 1) takes all flow reaching sink node 10 and returns it to source node 
1, thus restoring flow balance (for net demand by = 0). 

To finish a minimum cost network flow model, we need only introduce costs. 
Notice that the more the flow in return arc (10, 1), the greater the flow from source 
to sink. Thus we complete the model by placing a cost of —1 on flow in the return 
arc and taking all other costs as 0. Any minimum cost flow will necessarily maximize 
return arc, and thus source to sink flow. 


adding a return arc from sink to source with cost —1. All net demands and al 
_ costs on other arcs are zero. . 


Although principle | 10.32 | assures that maximum flow problems can be solved 
by minimum cost network algorithms, even more efficient special-purpose proce- 
dures have been developed. See the references at the end of this chapter for details. 


SAMPLE EXERCISE 10.18: MODELING MUM ELOWSASNETWORKS > 


Develop a minimum cost network flow model corresponding to the maximum flow 
model of Sample Exercise 10.17. 


Modeling: Following construction | 10.32], we add a return arc (4, 1) to obtain the 
following minimum cost flow problem: 
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Time-Expanded Flow Models and Networks 


As with the more general linear programs of Section 4.5, many applications of net- 
work flows involve time-expanded formulations to account for flows over time. This 
is especially true for those involving inventory management. 

In the specially structured network flow case, such problem lead to time- 
expanded networks. 


10.33 | Time-expanded networks model each node of a flow system as a series of 
nodes, one for each time interval. Arcs then re her flows between points 


ina particular time, or flows across time ina a particular location. 


EXAMPLE 10.5: AGRICO CHEMICAL TIME-EXPANDED 
NETWORK FLOW 


To see the idea, consider the case of Agrico Chemical,* which is a large chemical 
fertilizer company. Figure 10.15 sketches our fictitious version of the network model 
used to plan Agrico’s production, distribution, and inventory. Much in the spirit 
of OOI Example 10.1, products for this real company originate at 4 plants and are 
transshipped through 20 regional distribution centers before reaching customers in 
any of 500 service areas. 

Highly seasonal demand makes the Agrico case more complex than OOI. 
Agrico produces fertilizers throughout the year, but much of the demand comes 
in the spring quarter. Production capacity cannot accommodate spring demand in 
just one quarter. Thus the company builds up inventories at distribution sites during 
off-seasons to ship in the spring. Of course, storage is not unlimited, and inventories 
result in holding costs. 

Agrico’s decision problem requires choosing the amount to produce in each of 
the four quarters of the year, the pattern of shipping and storing it at distribution 
sites, and a scheme for sending it on to customers. We want to do all this at minimum 
total cost. 


Time-Expanded Modeling of Agrico Example 


The Agrico digraph of Figure 10.15 illustrates a time-expanded network. Total annual 
flow originates at the source node, but separate arcs representing production connect 
it to the 8 nodes (Pi, t) modeling plants P1 through P4 in quarters ¢ = 1,...,4. 
Capacities on those arcs would enforce the quarterly capacities of the correponding 
plants, and costs would reflect units costs of production. 

Plants in any quarter, ¢, are connected by transportation arcs to distribution 
center nodes (Dj, f) in the same quarter. Distribution centers, in turn, are linked to 
customer demands of the corresponding quarter. Costs on these arcs would reflect 
the unit cost of transportation, perhaps differentiated by the quarter in which the 
shipment takes place. 

Holding arcs between nodes for the same distribution center comprise the main 
new feature representable with time-expanded modeling. For example, the arc from 

D1, 2) to (D1, 3) models holding of the product at distribution site number 1 from 
the second to the third quarter. Its cost would be the unit holding cost at distribution 


4Based on E. Glover, G. Jones, D. Karney, D. Klingman, and J. Mote (1979), “An Integrated Pro- 
duction, Distribution, and Inventory Planning System,” Interfaces, 9:5, 21-35. 
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FIGURE 10.15 Agrico Time-Expanded Network Format 


site 1, and its capacity the size of the available storage. It would be impossible to 
model both flows in time and flow among facilities without distinguishing nodes by 
both place and time. 


A certain company can manufacture up to 15 thousand units of its product in any 
calendar quarter at a cost of $35 per thousand. The following table shows cost per 
thousand units shipped to each of the company’s two customers, and the number 
(thousands) of units demanded by each in various quarters. 


Demand by Quarter 


Shipping  ————__——__—— 
Customer Cost 1 2 3 4 
1 11 3] 9 2 f 


2 17 8 14 6 4 
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Assuming that inventory can be maintained at the plant for $8 per thousand per 
quarter, develop a time-expanded network flow model to determine the company’s 
dest production, distribution, and inventory plan. 


Modeling: Following | 10.33}, we create 4 nodes for the plant in different quarters 
and 4 nodes for each of the 2 customers in different quarters. Commodities for 
all quarters and customers arises at a common source node. Production arcs link 
commodities to plant nodes by quarter, and holding ares connect the plant nodes. 
Transportation arcs join the plant in each quarter to customer demands for that 
guarter. The result is the following time-expanded minimum cost network flow 
model: 


| 0.7 NETWORK SIMPLEX ALGORITHM FOR OPTIMAL FLOWS 


All the minimum cost network flow models of Sections 10.1, 10.5, and 10.6 are linear 
programs, so the LP algorithms of Chapters 5 and 6 could be applied. Still, we have 
already seen in Section 10.3 how the elegant structure of cycle directions leads to 
much more efficient computations in the network setting (principle ). 

In this section we fill in the details of rudimentary cycle direction Algorithm 
0A to derive one such procedure. It is known as the network simplex because it 
specializes the usual simplex computations of Chapter 5, taking advantage of the fact 
that simplex directions turn out to be cycle directions in the network flow case. A 
very different, cycle canceling method is detailed in Section 10.8. 


Linear Dependence in Node-Arc Matrices and Cycles 


Simplex search centers on bases—maximal collections of linearly independent col- 
umns drawn from the constraint matrix of main (standard form) constraints Ax = b. 
Columns of the main constaints in network flow models are columns of the node- 
arc incidence matrix [format (10.4)]. They correspond to arcs. Thus, to understand 
simplex in the network context, we must first understand what collections of arcs 
correspond to linearly independent collections of columns. 
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Begin with cycles. For example, consider the OOI cycle (2, 3), (1, 3), (1, 4), (2, 4) 
illustrated in Figure 10.16. The figure displays both the cycle and the corresponding 
columns of the node-arc incidence matrix shown in Table 10.1. 


Arc 
Node (2,3) (1,3) (1,4) (2,4) 

1 0 -1 -1 0 

2 -1 0 0 -1 

3 +1 +1 0 0 

4 0 0 +1 +1 

5 0 0 0 0 

6 0 0 0 0 FIGURE 10.16 Cycle and the 

: ; : j Corresponding Node—Arc Incidence 


Columns of OOI Example 


Suppose that we apply weights of +1 on all column vectors for all forward arcs 
in the cycle, and —1 on columns of reverse arcs. Then 


0 —1 —1 0 0 

—1 0 0 sl 0 

+1 +1 0 0 0 

0 0 aril +1 0 

+1 a fae 1 0 apa o |~ 1 ‘oul Galan 
0 0 0 0 0 

0 0 0 0 0 

0 0 0 0 0 


Our +1 weighted sum of the columns has produced the zero vector. This implies that 
the columns are linearly dependent because any one of the vectors can be expressed 
(by transposing) as a nonzero linear combination of the others. 

The +1 weights we have chosen are exactly those of the corresponding cycle 


direction| 10.11 | . Thus for the same (Figure 10.5) reasons that cycle directions satisfy 
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AAx = 0, such a +1 linear combination of node—arc incidence columns for the arcs 
of a cycle will always yield the zero vector. 


ce matrix columns for arcs « cycle forma linearly de- 


Since column vectors of a basis must be linearly independent, we have an immediate 
consequence: 


10.35 | Basic sets of arcs for minimum cost network flow models can 
cycles. - : es 


Cycles are not the only way to produce linearly dependent sets in network 
flow models. For example, return to the complex direction of Figure 10.8. That 
weighting of node—arc incidence columns also produces the zero vector because the 
direction satisfies AAx = 0. Thus the collection of arcs with nonzero weights is 
linearly dependent. 

Still, we know from principle | 10.17 | that complex directions like that of Fig- 
ure 10.8 decompose into weighted sums of cycle directions. This implies that each 
includes at least one cycle. 


del 


Consider the following digraph: 


Determine which of the following arc sets could be part of a basis for the correspond- 
=z node-arc incidence matrix. 


a) {(1, 2), (4, 3)} 

D) {(2, 4), (4, 3), G, 2)} 

©) {d, 2), 1,4), 4, 3), 2, 9} 
d) {(1, 2), 2, 4), G, 2)} 
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Analysis: 


(a) This set is linearly independent under principle 10.36 | because it contains no 
cycle. It could be part of a basis. 


(b) These arcs form a cycle. Thus, under principle /10.35 |, they cannot be part of a 


basis. 
(c) This set is linearly dependent because it contains cycle 1-2-4-1. Under principle 
10.35 | , it cannot be part of a basis. 


cycle. It could be part of a basis. 


Spanning Trees of Networks 


We say that a graph is connected if there is a chain between every pair of nodes. A 
graph is a tree if it is connected and contains no cycles. A tree is a spanning tree if it 
touches every node of a graph. 

Figure 10.17 illustrates these definitions. Part (a) shows a spanning tree. It 
is connected, it contains no cycles, and it touches all nodes of the OOI network. 
Parts (b) to (d) demonstrate the various ways that graphs can fail the definition of a 
spanning tree. 


(b) Not connected 


re 


(c) Not spanning (d) Contains a cycle 


FIGURE 10.17 Notion of a Spanning Tree 
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The connected but cycle-free nature of trees leads to an important property: 


For example, nodes 8 and 4 are connected in the spanning tree of Figure 10.17(a) by 
the chain 8-1-3—7-4 and by no other. If there were two such chains, they would form 
a cycle, and there must be one if the tree is connected. 


Spanning Tree Bases for Network Flow Models 


Returning to the issue of bases for node—are incidence matrices, the example of 
Figure 10.17(d) has no hope of being a basis because it contains a cycle. Correspond- 
ing columns of the node-arc incidence matrix will be linearly dependent (principle 
10.34|). The graphs in parts (a) to (c) have no such cycles. Thus principle 
assures us that corresponding columns are linearly independent. But are they a basis? 
A basis must be a maximal linearly independent set (i.e., it should be impossi- 
dle to enlarge it without creating a dependency). Notice that the examples in Figure 
10.17(b) and (c) fail this test. We know from principle that a dependency will 
be created only if we form a cycle. That cannot occur when we insert an arc between 
previously unconnected nodes. For example, adding arc (4, 3) in Figure 10.17(b) pre- 
serves linear independence. The same is true for adding arc (3, 5) in Figure 10.17(c). 
The spanning tree of Figure 10.17(a) is different. Because there is already a 
unique chain between every pair of nodes (property ), inserting any other arc 
closes a cycle. For example, adding arc (2, 4) closes cycle 2-4-7-3-2. 


10.38 | Adding an ate toa spanning tree produces a unique cycle. 


It follows immediately that spanning trees correspond to maximal linearly in- 
dependent sets and so bases. We have reached the key to understanding simplex 
computations in minimum cost network flow models: 


Seeimine which of tine four arcs 5 sets i in ele Exercise 10. 20 form a basis for the 
corresponding network flow problem. 


Analysis: We have already seen in Sample Exercise 10.20 that sets (b) and (c) are 
inearly dependent. Set (a) is linearly independent, but it is not a basis under princi- 


ole| 10.39 because the arcs do not forma spanning tree. Set (d) does form a spanning 
tree. Thus it is the only basis of the four. 


524 


Chapter Ten e Network Flows 


Network Basic Solutions 


Simplex algorithms proceed from basic feasible solution to basic feasible solution. 
For the network case, where capacities or upper bounds are present, we define basic 


solution in the upper-/lower-bound sense of Section 5.9 (principle ). 


10. 40 | In basic so ems, nonbasic ares have flow 
— to eithe : he unique flow achieving flow _ 
balance for the nonbasic val = s specified, ‘The flow is basic feasible of all basis 
_ flows are within bound : . 


Figure 10.18 illustrates for the OOI example of Figure 10.2. The corresponding 
basis is highlighted. There is positive flow on nonbasic arc (4, 3), but it equals capacity 
u43 = 25. Only basic arcs can have flow x;; other than zero or uj;. 

To compute this basic solution, we first assign flow 0 to all nonbasics except 
(4,3) and 25 to x43. Only one choice of basic flows can then meet net demand 
requirements at each node. For example, with nonbasic flow x3,7 = 0, the only flow- 
balancing choice for basic flow x47 = 610. The solution of Figure 10.18 is basic 
feasible because all such basis flows are within bound limits. 


SAMPLE EXERCISE 10.22: COMPUTING NETWORK BASIC SOLUTIONS 


In the following network, numbers on nodes indicate net demands b; and those on 
arcs show capacities uj. 


10 @) 1) 30<—~» 


For each of the following choices on nonbasic arcs, compute the corresponding basic 
solution and determine whether it is basic feasible. 


(a) (1, 4) nonbasic lower-bounded, (2, 4) nonbasic upper-bounded 


(b) (3, 4) nonbasic lower-bounded, (1, 2) nonbasic upper-bounded 
Analysis: We apply principle | 10.40]. 


(a) Here the basis is {(1, 2), (1, 3), G,4)}. With x14 = 0 and x24 = 20, the unique 
choice of basic values that meets net demand requirements at all nodes is x12 = 
1,3 = 20, x3,4 = 10. This basis solution is not basic feasible because flow x1,2 exceeds 
capacity 1,2 = 10. 


10.7 Network Simplex Algorithm for Optimal Flows 525 


FIGURE 10.18 Initial Basic Solution for OOI 
Example 


(b) Here the basis is {(1, 3), (1, 4), (2, 4)}. With X34 = 0 and x;,2 = 10, the unique 
choice of basic values that meets net demand requirements at all nodes is x13 = 
%2,4 = 10, x14 = 20. Since all basic flows are within bounds, this basic solution is 
basic feasible. 


Simplex Cycle Directions 


Simplex directions (construction | 5.21 | [S24 of Section 5.3) in linear programs are formed 
by increasing one nonbasic variable from its lower bound (or decreasing a nonbasic 
‘rom its upper bound) and changing basic variables in the unique way assuring that 
main constraints remain satisfied. For network flow cases, every basic set of columns 


arcs) forms a spanning tree (principle | 10,39|). Thus introducing any nonbasic arc 


creates a unique cycle (principle| 10.38 |), and the corresponding cycle direction must 
de the simplex direction. 


as that arc. The 16 simplex direction fo ee a onbasic: are at ‘capacity 
flow u;; is the cycle directior 


Simplex cycle directions need not be feasible because some decreasing basic 
low may already equal zero or some increasing basic flow may be at capacity. As 
with other simplex methods, however, we may ignore such degeneracy without much 


practical effect except step sizes 4 = 0 (principle ): 
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CTING NETWORK SIMPLEX DIRECTION 


Consider the follanae. oe Numbers next to nodes are net demands b,;, and 
those on arcs are cost, capacities, and current flows (Cij, Uij, Xij)- 


(G u, x) 


Taking highlighted arcs as the basis, construct the simplex directions for all nonbasic 
arcs. 
Analysis: We apply construction [10.41]. Introducing nonbasic arc (1, 3) produces 
unique cycle 1-3—2-1. Since nonbasic arc (1, 3) is presently at flow 0, the corresponc- 
ing simplex direction will be the cycle direction for 1-3-2-1 with (1, 3) a forward are 
That is, Ax,,3 = +1, Ax23 = Axi. = —1. 

Nonbasic arc (3, 4) is presently at capacity. To find its cycle/simplex directior 
we pass the unique cycle it forms in the basis so that (3, 4) is a reverse arc. That is 
we employ cycle direction 4-3-2-4 with Ax34 = Ax23 = —1, Axz4 = +1. 


Network Simplex Algorithm 


Algorithm 10B combines these ideas in a special simplex version of cycle direction- 
based network search, Algorithm 10A. Instead of considering all possible cycle d 

rections at each iteration, we check only the simplex directions induced by the unique 
cycles for increasing or decreasing nonbasic arcs. Simplex theory for all linear pro- 


grams (principle of Section 5.3) tells us that these directions are sufficient t 
find an optimal solution. 

Comparison of this algorithm statement with the general simplex computations 
of Chapter 5 will highlight the convenience made possible by the special networs 
flow structure. To find simplex directions and determine whether they improve the 
objective function, we need only trace our way through the basis tree. 


Network Simplex Solution of OOI Example 


Figure 10.19 details solution of our OOI example by Algorithm 10B, starting from 
the initial basic feasible solution in Figure 10.18. At the first iteration, the simple 
directions for the 7 nonbasic arcs are as follows: 


Nonbasic Cycle c 
Increase (1, 3) 1-3-2-8-1 3 
Increase (2, 4) 2-4-1-8-2 -1 
Increase (3,4) 3-4-1-8-2-3 via (3,4) —4 
Increase (3, 7) 3-7-4-1-8-2-3 8 
Increase (4,3) 3-4-1-8-2-3 via (4,3) —4 
Increase (4, 5) 4-5—3-2-8-1-4 8 
Increase (4, 6) 4-6-3-2-8-14 6 
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t ae ine N AOD : i Ad 


Step 0: Initialization. Choose any starting basic feasible flow x, identify 
the corresponding basis spanning tree, and set solution index t < 0. 

Step 1: Simplex Directions. For each nonbasic arc, examine the simplex 
direction associated with the unique cycle that arc forms in the basis spanning 


tree and apply test | 10.14 | to determine whether the direction improves. 


Step 2: Optimal. If no simplex cycle direction improves, stop; flow x is 
globally optimal. Otherwise, choose as Ax some improving simplex cycle 
direction, and let (p, g) denote the corresponding nonbasic arc. 

Step 3: Step Size. Compute the maximum feasible step A in direction Ax 


(rule [10.15)): 


Ay <— min{ (uj; — x{) : (i, j) forward} (+o if none) 


Oo 
ij 
A <— min{At, a7} 


A7 < min{x"? : (i,j) reverse} (+00 if none) 


If A = +00, stop; the model is unbounded. 
Step 4: Advance. Update 


xD — x + Ax 


by increasing flows on forward arcs of the cycle direction and decreasing 
those on reverse arcs by the amount A. 

Step 5: New Basis. |f some arc other than (p, q) established the minimum 
A in step 3, replace any such arc in the basis spanning tree by (p,q). 
Increment t — t+ 1 and return to step 1. 


Any simplex direction with ¢ < 0 could provide a means of improving flow x). 
Figure 10.19 employs the one associated with increasing nonbasic, lower-bounded 
are (2, 4). The unique cycle formed in the basis tree immediately identifies the rest 
of the associated cycle direction. Arcs (2,4) and (1,8) increase; (1,4) and (2, 8) 
jecrease. Thus A+ = oo, A~ = min{75, 635}, and A = 75. Arc (2, 8) established the 
_limit, so (2, 4) replaces (2, 8) in the basis. Updating produces flow x of Figure 
10.19. Notice that the new basis is also a spanning tree. 

The simplex direction decreasing nonbasic, upper-bounded arc (4, 3) now im- 
oroves for x‘. The unique cycle formed by (4, 3) in the current basis yields a cycle 
direction decreasing (4,3) and (2, 4) while increasing (2,3). Move limit 4 for this 
direction occurs when nonbasic (4, 3) reaches its lower bound of 0. Thus flows are 
spdated, but the basis remains unchanged. 

Flow x) in Figure 10.19 is the result. Increasing nonbasic (3, 4) can improve 
this flow because @ = —3 on the corresponding simplex direction. Adjustment of 
. = 25 around the implied cycle direction improves to flow x®. Again incoming arc 
3. 4) itself stops improvement, so the basis remains unchanged. 

Comparison to Figure 10.3 will show that new flow x®) is now optimal. None 

f the 7 simplex cycle directions is improving. 
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Flow x Simplex Direction Ax 


(2, 4) increass 
e=-1 


(4, 3) decreas: 
t=-3 

Mt = +00 

x =25 

A=25 

no basis chans 


(3, 3) increase 
eae 


A= 2s 
Z; x =50 
We OS) 


no basis charms 


No improving simplex directions 
exist. Flow x©) is optimal. 


FIGURE 10.19 Network Simplex Solution of OOI Example 


SF a a a a a a a a 
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SAMPLE EXERCISE 10.24: APPLYING THE NETWORK SIMPLEX ALGORITHM 


Return to the network flow example and starting basis of Sample Exercise 10.23, and 
compute an optimal solution with network simplex Algorithm 10B. 


Analysis: At the first iteration, increasing (1,3) produces a simplex direction for 
cycle 1-3-2-1 having ¢c;3 = 2 — 4 — 6 = —8. Decreasing nonbasic (3, 4) yields the 
direction for cycle 4-3-2-4 with ¢34 = —5 — 4+ 13 = 4. Thus we update along the 
former by 4 = 10, and replace (2, 3) in the basis by (1, 2) because (2, 3) established 


The new solution is 


9; 
oy 


Ge u, X) 


Now the cycle direction for increasing (2, 3) has ¢2.3 = 8 and that of decreasing (3, 4) 
had ¢3.4 = 12. Neither is improving, and the latest flow is optimal. 


] 0.8 CYCLE CANCELING ALGORITHMS FOR OPTIMAL FLOWS 


Algorithm 10A in Section 10.3 outlines the common computational logic of many 
network flow algorithms. The big question left unanswered is: How do we iden- 
tify improving feasible cycle directions at each iteration, or prove that none ex- 
ist? In Section 10.7 we presented one method based on the simplex algorithm. In 
this section we detail a quite different approach: cycle canceling by shortest path 
computation. 


Residual Digraphs 


The cycle canceling method (and many other network flow procedures) begins each 
iteration by constructing a residual digraph that details available options for improv- 
ing feasible cycle directions. Each arc in the original digraph yields up to two in the 
residual one, depending on whether its present flow can feasibly increase or decrease 
or both. 


for each increasable a arc flo y vx 


with cost —¢);, for each decreasable are flow x05 — a= 
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Figure 10.20 illustrates for the OOI starting conditions depicted in Figure 10.9. 
Every flow that can both increase and decrease yields two opposed arcs in the resid- 
ual digraph. For example, are (2,3) with current flow x) = 1000 produces both 
residual graph arc (2,3) and arc (3,2). The first, increase arc represents the fact 
that x23 can feasibly become larger. Its cost is c2,3 = 4, the unit cost of such a flow 
increase. Decrease arc (3,2) shows that the flow can also become smaller without 
losing feasibility. It has cost —c2,3 = —4, the unit savings from a flow decrease. 

Flows that cannot both increase and decrease produce just one arc in the resid- 
ual digraph. For example, flow ay) = 0. It can only increase, so the residual graph 
has only arc (1, 4) at cost cy4 = 8. 


SAMPLE EXERCISE 10.25: CONSTRUCTING RESIDUAL DIGRAPHS 


Consider the network flow problem sketched below. Numbers on nodes show net 
demand b;, and those on arcs show costs, capacities, and current flows (cj, Ujj, Xij)- 


(5, 100, 0) 


-100 G)o-~» 


oe pdt (c, u, x) 


0 @ G) 100 


Construct the corresponding residual digraph. 


Analysis: Applying construction | 10.42, the residual digraph is 


(10, 200, 90) 
(8, =, 0) 


Feasible Cycle Directions and Dicycles of Residual Digraphs 

The point of constructing a residual digraph is to make it easier to identify improving 
feasible cycle directions in the original graph. Notice first that any cycle direction 
meeting principle} 10.13 |’s requirements for feasibility—forward flows below capac- 
ity, reverse flows positive—now corresponds to a dicycle (definition | 10.10} ) in the 
residual graph. A backward, decrease arc has been provided for each reverse arc 07 
the cycle. 
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: S 
Ke FIGURE 10.20 Residual Digraph for 


OOI Example Flow of Figure 10.9 


10.43 | Cycle direction Ax is feasible for a current flow x if and only if the 
residual graph for x“ contains a dicycle with forward arcs of the Ax cycle cor- 
responding to increase arcs in the residual dicycle, and reverse arcs of the cycle 


One example is cycle 7-3-1-4—7 of the original digraph. 


Q--@ @ 


x 
forward \ 6; 
1 Ge 


oe. ey 


(a) Original graph (b) Residual digraph 


The corresponding cycle direction is feasible because forward arcs (1, 4) and (4, 7) 
are below capacity, and reverse arcs (3,7) and (1, 3) have positive flow. It follows 
that each forward arc of (a) corresponds to an increase arc in (b), and each reverse 
arc of (a) yields a (backward) decrease arc in (b). 


Improving Feasible Cycle Directions and Negative Dicycles 
of Residual Digraphs 


We seek improving feasible cycle directions, those with negative reduced costs: 


¢ = (total forward arc cost) — (total reverse arc cost) 


But under construction [10.42], forward arcs appear in the residual digraph as in- 
crease arcs with the same cost, and reverse arcs appear as backward, decrease arcs 
with the sign of their costs switched. It follows that the ¢ of a feasible cycle direction 
is exactly the length of the corresponding dicycle in the residual digraph. Dicycles 
of negative length yield what we require. 
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10.44 | Feasible cycle direction Ax is improving if and ony - the cep 
residual graph dicycle is a negative dicycle. 


Again we can illustrate with the direction for OOI example cycle 7-3-1-4-7. 


"@ 


(a) Original graph (b) Residual digraph 
The original cycle is improving because the corresponding dicycle has negative total 
length. 


SAMPLE EXERCISE 10.26: CONNECTING ORIGINAL AND RESIDUAL DIGRAPHS 


Return to the example of Sample Exercise 10.25. 
(a) Show that the direction for cycle 1-3-4—2-1 is both feasible and improving. 


(b) Identify the corresponding dicycle in the residual digraph, and verify that it is a 
negative dicycle. 


Analysis: 


(a) The direction for cycle 1-3-4—2-1 is feasible because forward flows (1, 3) and (3, 4 
are below capacity, and reverse flows (2, 4) and (1,2) are positive. It is improving 
because 


ce = (5+ 8) — (17+ 10) = —-14 


(b) The residual digraph of Sample Exercise 10.25 contains dicycle 1-3-4—2-1 with 
increase arcs (1, 3) and (3, 4), plus decrease arcs (4, 2) and (2, 1). Its total length is 


53+8-17-10=-14=c¢ 


Using Shortest Path Algorithms to Find Cycle Directions 


Principles | 10.43 | and | 10.44 | reduce our search for improving feasible cycle direc- 
tions to a hunt for negative dicycles in the residual digraph. The advantage is that we 
already know algorithms for identifying negative dicycles or for proving that none 
exists. 

The Floyd—Warshall shortest path Algorithm 9B, which we derived in Section 
9.4, is one. That procedure is designed primarily to compute shortest paths from all 
nodes of a graph to all other nodes. If the given graph contains a negative dicycle. 
shortest path computation fails, but a negative dicycle is returned. Interchanging 
roles to make negative dicycles the normal outcome and completion of shortest path 
calculations the exception produces exactly the improving feasible cycle direction 
subroutine that we require. 


10.8 Cycle Canceling Algorithms for Optimal Flows 65 


10.45 Application of Floyd-Warshall Algorithm 9B to the residual digraph for 
a current oe flow either yields a negative dicycle, indicating that the di- 
fb or completes a shortest path computation, proving th that improving 
feasible cycle directions do not exist. 


SAMPLE EXERCISE 10. ai FINDING DIRECTIONS WITH FLOYD-WARSHALL 


Apply Floyd- _Warshall Algorithm 9B to the residual digraph of Sample Exercise 
10.25 either to produce an improving feasible cycle direction or to prove that none 
exists. 


Analysis: After its third main iteration, Floyd—Warshall computation produces the 
following shortest path and inbound node results: 


vOrk, L\ 
kp G=Nee C2 eS 
1 0) 10 5 
2 -10 0 -5 
3 lene) [o.e) 0 
4 -27 -17 22 


Computation halts with the conclusion that there is a negative dicycle because diag- 

nal distance v®)[4, 4] = —14 is negative. Using d[k, £] labels to recover the dicycle, 
4,4] = 3, d[4,3] = 1, d[4, 1] = 2, d[4,2] = 4. Thus 4-2-1-3-4 is a negative di- 
cycle of the residual graph. We already know from Sample Exercise 10.26 that the 
corresponding cycle direction is indeed improving and feasible. 


Cycle Canceling Solution of the OOI Example 


Algorithm 10C formalizes principle} 10.45 |in the cycle canceling method for network 
jow optimization. Figure 10.21 details its application to our OOI example of Figure 
10.9. 

As before, graphs on the left of Figure 10.21 show flows in the real network. 
Notice that the sequence of solutions differs from that of Table 10.10 because different 
mproving feasible cycle directions were used. Still, both finished at the same optimal 
flow. 

Graphs at the right in Figure 10.21 depict the residual digraphs for each flow 

acountered. Details of Floyd—Warshall computation have been omitted, but the 
segative dicycles obtained are highlighted. For example, the first application of 
Algorithm 9B identified negative dicycle 7-3—-1-4~7 in the residual digraph for initial 
Sow x). Its total length is -17 -7+8+5 = —11. Application of rule| 10.15 | around 
the corresponding cycle in the real OOI network yields A = 560. Then increasing 
tows by 560 on forward arcs (1, 4) and (4, 7), while decreasing by the same amount 
a reverse arcs (3, 7) and (1, 3) produces the indicated flow x), 
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Step 0: Initialization. Choose any starting feasible flow x, and set solution 
index t <— 0. 
Step 1: Residual Digraph. Construct the residual digraph corresponding 


to the current flow x (principle | 10.42 |). 


Step 2: Floyd-Warshall. Execute Algorithm 9B on the current residual 
graph. If Floyd-Warshall computation terminates with no indication of a 
negative dicycle, stop. No improving feasible cycle directions exist, and 
current flow x® is globally optimal. 

Step 3: Cycle Direction. Use Floyd—Warshall decision labels d[k, ¢] to trace 
a negative dicycle in the residual graph, and construct the cycle direction 
Ax for the corresponding cycle in the original graph. 

Step 4: Step Size. Compute the maximum feasible step 4 in direction Ax 


(rule [10.15}): 


At <— min{(uij — x 
ATS min{x;) 


h <—min{rt,a} 


(2) 


Fy) Gad) forward} (-+-oo if none) 


: (i, j) reverse} (+00 if none) 


If 1 = 00, stop; the model is unbounded. 
Step 5: Advance. Update 


xi) ore x 4+iAx 


by increasing flows on forward arcs of the cycle direction and decreasing 
those on reverse arcs by the amount 4. Then increment ¢ — ¢+1 and return 
to Step 1. 


Computation continues until t = 4. There Floyd—Warshall terminates on the 

residual digraph without discovering a negative dicycle. We may conclude (principle 

10.45 1) that no negative dicycle exists in the residual digraph, so that no improving 
feasible cycle direction exists for our current flow. The flow must be optimal. 


SAMPLE EXERCISE 10.28: APPLYING THE CYCLE CANCELING ALGORITHM 


Apply Cycle Canceling Algorithm 10C to the network flow problem of Sample 
Exercise 10.25. 


Analysis: We have already seen in Sample Exercise 10.27 that Floyd—Warshall com- 
putation on the initial residual digraph yields negative dicycle 4-2-1-3-4. Forware 
arcs in the corresponding cycle imply that A* = min{(100 — 0), (co — 0)} = 100 
Reverse arcs give A~ = min{90, 90} = 90. Thus step size 4 = min{100, 90} = 90. 

Adjustment by this amount around cycle 4-2-1-3-4 produces the following 
new flow and residual digraph: 


i ee et ee ee Le ee eS ee 
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t Flow x Residual Digraph 


FicurRE 10.21 Cycle Canceling Solution of the OOI Example 


Step 
e==11 
Xt = +00 
x = 560 
A = 560 
c=-l 
M = 25 
x = 560 
A =25 
c=-11 
Agi = +00 
Meo S 
Ana 25 
z=-9 
Any = +00 
Aga) 
4 =25 


Soo 
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t Flow x Residual Digraph Step 


optimal 


FIGURE 10.21 Cycle Cancelling Solution of the OOI Example (Continued) 


(5, 100, 90) 


-100 G) 0<—~b 


20, 90) 


ce _ (6 U, x) 
: @) (17, 90, 0) ® 5d 


Floyd—Warshall computation on the new residual digraph will show that there is nc 
negative dicycle. Thus the revised flow is optimal. 


8 


‘ oo MULTICOMMODITY AND GAIN/LOSS FLOWS 


There is almost always a tradeoff between the tractability of operations research 
models and the span of applications to which they can be adapted. Networks are no 
exception. In this final section we explore briefly some more broadly applicable net- 
work forms that retain part, but not all, of the tractability of minimum cost network 
flows. 


Multicommodity Flows 


An implicit assumption in our previous flow models has been that all flows are in 
a single commodity. In the OOI example of Section 10.1, it was toaster ovens. In 
other examples the commodity was people, chemicals, or manufacturing tasks. In 
every case we assumed that demands at sink nodes could be filled from any suppl) 
node with a path to the sink. That is, we assumed that flows were interchangeable or 
fungible. 

Multicommodity flow problems arise when flows passing thorough a commoz 
network must be kept separate; flow is not fungible. 


10.46 | Multicommodity flow models 


commodities are moving through a common ne 


EXAMPLE 10.6: BAY FERRY MULTICOMMODITY FLOW 


As usual, it will help to think of a simple (fictitious) example. Figure 10.22 depicts 
traffic flows in communities around an ocean bay. Each morning, the population of 
the three residential communities travels to the two industrial and two commercial 
centers in the region. Table 10.3 details flows by origin and destination. For example, 
1250 of the 6000 daily trips originating at residential node 4 have industrial park node 
7 as their destination. 

At present, geography limits each trip to a single path. Numbers on arcs of 
the network in Figure 10.22 show the distance in kilometers between various points. 
For instance, those originating at node 1 and bound for node 7 must drive all the 
way around the bay through nodes 2 to 6. The 21,100 trips arising daily in the three 
residential communities produce a total of 399,250 kilometers of driving along such 
routes. 

Regional planners are considering various improvements to reduce air pollu- 
tion by reducing the number of kilometers driven. One idea is the ferry indicated by 


residential 


industrial 


commercial 


residential 


25.0 
FIGURE 10.22 Bay Ferry Example Network 


TABLE 10.3. Daily Trips in the Bay Ferry Example 


ips by Destinati 
Trip Origin Total Zopsiby Deshaakow 


Node Trips 1 2 3 4 | 6 7 


it 2,850 900 750 40 10 600 550 
4 6,000 100 2000 1100 — 150 1400 1250 
5 12,250 110 4000 2200 200 — 3300 2440 
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dashed arcs (2, 6) and (6, 2) in Figure 10.22. If a ferry were introduced, it could carry 
2000 cars in each direction during the morning rush period. We want to know how 
many kilometers of driving might be saved. 


Multicommodity Flow Models 


Clearly, Figure 10.22 depicts a flow network. But what is flowing? If we treat all trips 
as equal, we have only a single commodity. Then, however, it would be feasible to 
fulfill the demand for 1250 trips from origin node 4 at sink node 7 with trips from 
any source. Naturally, a minimum distance solution would prefer ones from nearby 
source 5, leaving demands for node 5 trips at locations 2 and 3 to be filled from the 
closer source 1. Such a solution makes no sense for the application because trips are 
not fungible. 

We must form separate commodity networks for trips from each of the three 
sources. That is, we must model in multiple commodities. Still, the commodities 
are not independent. All share the 2000-trip capacity of the proposed ferry. Such 
interdependencies are typical of multicommodity flows. 


arately because ae interact through shared arc capacities. 


Suppose that we employ constants 
Cq.ij 2 unit cost of commodity g flow in arc (i, j) 
uj; 4 shared capacity of arc (i, j) 
bax & net demand for commodity g at node k 
and the decision variables 


Xq,ij & commodity g flow i in arc (i, /) 


10.48 | The multicommodity network flow w model ona digraph with nodesk eV 
and arcs (i,j) € A is 


min 5 S outat 


 @ (jeA : 
Sit. == oo ve Koki = Dax forallg,k eV 
GK)EA (pea ee . 
oe Xq.ij S Uij === Ota) = A 4 
os SECS eee ei oi Te ees = oe 
“1 2 8 forallg,@) <A 


Table 10.4 presents the corresponding formulation of our Bay Ferry example. 
There commodity 1 corresponds to flows from origin node 1, commodity 2 denotes 
flows from residential node 4, and commodity 3 relates to residential node 5. Notice 
that there are separate systems of flow conservation equations for each commodity. 
plus a common set of capacity constraints on the two arcs with flow limits. 


10.9 Multicommodity and Gain/Loss Flows 539 


TABLE 10.4 Bay Ferry Example Model 


min 3.5x4 49+ 3.5x1 24 + 3%123 + 3X1 39+ 5% 3.44 5143 (minimize driving) 
+ 15x, 45+ 15x41 5.44 4156+ 441,654 2.5%167+ 252X176 

aE 3.5x7 12 + 3.5x224 co 3x23 ap 3x23, + 5x93.4 ap 5x2,43 

rte 153x245 os 15x25 4 + 4X 5,6 fe 4x9 65 + 2.5%2,6,7 =f 2.5X2.76 

+ 3.5319 + 3.5x391 + 3x3.93 + 3x3 39+ 5X3.3.4 + 5X3.43 

+ 15x3.45 + 15354 + 4%3.56 + 4%3.65 + 2.9%3,.6,7 + 2.54376 


St 21-112 = -2,850 (commodity 1) 
Ayia ase tT 62> M193 =" 12.65 900 
423+ %143 —*13,27%134 = 750 
134+ 15,4 —*1437 1,45 = 40 
X1 45 +1657 *1547 *15,6 = 10 
X126 + %15,6+ %1,7,6 + *1,62-%165-*1,67 = 600 
*1,6,7- *1,7,6 = 550 
%221—*2,1,2 = 100 (commodity 2) 
Xo42+%232 + %262-%221-%223-%226 = 2,000 
793+ X2,43 —*2,3,2- %2,3,4 = Perils to) 
X34 + ¥2,5.4 —%2,43-%2,45 =  -6,000 
%7.4,5 + X2,6,5 — %2,5,4— %2,5,6 = 150 
X9,2,6 + %2,5,6 + 2.7.6 + ¥2,6,2-%2,65-%267 = 1,400 
*2,6,7 — *2,7,6 SAS 
%3,2,1- 31,2 = 110 (commodity 3) 
X39 + %33,2 + X36,2-%32,1- 4323-4326 = 4,000 
393+ 343 —%33,2—*33,.4 = e2-200) 
X3.3.4 + 35,4 —%3.43— *3.4,5 = 200 
X3.4,5 + X3.6,5 —*35,4— *3,5,6 = 12,250 
X3,2,6 + X3,5,6 + X3.7,6- %3,62—%3,65-%3,6,7 = 3,300 
X3,.6,7 — *3,7,6 = 2,440 
12,6 + 2.2.6 + 3.2.6 < 2,000 (capacities) 
X16,2 + X2.6.2 + %3,6,2 < 2,000 
all x, ;;2 0 


An optimal solution reduces the total driving to 280,770 kilometers, a savings 
of 29.7%. This is accomplished by accommodating x1,2,6 = 1160 commodity 1 trips 
on the 2-to-6 ferry, and x26 = 840 commodity 2. In the reverse direction the ferry 
carries X3,6,2 = 2000 commodity 3 trips. 
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Labels on arcs show costs for three commodities and common capacity (¢1,i;, C2; 
C3,;j, Uij). Labels on nodes show net demands (b,x, b2,x, b3,). Formulate the corre- 
sponding multicommodity network flow model. 


Modeling: Following format | 10.48 |, the model is 


MM. 4,19) 43.2 fF X21 X903 te 2,30 


TeXOM tr 03 1. sete 05, 312 ale oe 


Sit. 21 Oger X31 — Mi — 0143 eee L 
Xia Misia — Mog = X4,25"— 20 
Hyg M,3 — Mise =X ao9—7 1 
Xo OM 0 31 — X19 — Ao, 1 e 
X2,1,2 + X2,3,2 — X2.2,1 — X2,23 = —1 
X2,1,3 + X2,2,3 —%2,3,1 — X23,2 = 0 
X32,1 + 331 —%312—%313= 0 
X3 fie X35'9 030.1 — Xai 
¥3,1,3 + %32,3 — 43,41 — %33,2——LL 
X12 X22 FP 43,1.2 Ss: E 
XA 2 Shcte 22.3) te 8123 << pill 
X1,3,1 + %2,3,1 + 3,3,1 ss Pil 


all Xqij 2 0 


Tractability of Multicommodity Flow Models 


Multicommodity flow formulations are certainly linear programs. In fact. 
they are rather special linear programs for which unusually efficient algorithms can 
be developed. Also, we continue to think in terms of flows and represent complex 
models in simple diagrams. 

Still, very little of the elegant structure presented for single-commodity cases 
in Sections 10.2 to 10.5 carries over to the multicommodity setting. Most important 


is a loss of integrality property | 10.20). 


flow ae may be fractional. 


The instance of Sample Exercise 10.29 illustrates. Each commodity is supplied 
at one node and demanded at another. Since costs on the inner dicycle 1-2-3-1 are 
zero, all commodities compete for the unit capacities on those arcs. It is easy to verify 
that the unique optimal solution makes 


= vos = 1s 


X2,2,3 = X2,3,1 = X2,2,1 = 


NI NI NI 


X3,3,1 = %3,1,2 = 43,3,2 = 


3 


Total cost of this fractional solution is 5, versus 2 for any all-integer flow. 


10.9 Multicommodity and Gain/Loss Flows 


Flows with Gains and Losses 


Another implicit assumption of network models so far encountered is that the num- 
ber of units entering one end of any arc is the same as the number of units leaving 
the other end. Many realistic modeling situations fail this assumption. Power distri- 
bution networks lose electricity along distribution lines, groundwater seeps into and 
sut of sewers along their runs, and invested funds earn interest as they flow through 
time. All such circumstances produce flows with gains and losses. 


10.50 | Network flow problems with gains and losses model circumstances where 
a given constant aj; units of flow exit arc (i, j) at node j for every 1 unit of flow 
entering ati. Value a;; > 1 indicates a gain, a;; mplies a loss, and a;; = 1 
yields ardinary network flows = = =. 


EXAMPLE 10.7: TINYCO CASH FLOW WITH GAINS AND LOSSES 


One of the most common settings for flows with gains and losses is financial transac- 
tions. A simple case is cash flow modeling. 

Cash flow management deals with cash and near equivalents such as short-term 
oonds. The object is to have cash available when it is needed to pay obligations while 
earning as much interest as possible on funds that are not needed immediately. In 
some cases it is also possible to borrow cash against future receipts. 

Figure 10.23 presents a specific instance for a fictitious company we will call 
Tinyco. Nodes 1 to 5 represent cash over time, with numbers b, next to each node 
showing net demand for cash by month (in thousands of dollars). Nodes 6 and7 model 
funds invested in short-term bonds, starting from an initial holding of $200,000. 


-100 


300 —500 -10 30 spot 


0.9901 


[0.998 | 


200 +b 


So 
S 


FIGURE 10.23 Tinyco Example Network 


We assume that cash can be invested to return 0.5% per month, and bonds 0.9%. 
This produces the gain multipliers a; ;,, shown in boxes on arcs in Figure 10.23 running 
forward in time. For example, arc (3, 4) carries multiplier a3,4 = 1.005 because each 
dollar invested at month 3 yields 1.005 dollars after a month’s interest at 0.5%. 


5Based on B. Golden and M. Liberatore (1979), “Models and Solution Techniques for Cash Flow 
Management,” Computers and Operations Research, 6, 13-20. 
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Reverse arcs along the cash part of Figure 10.23 illustrate loss arcs by modeling 
borrowing. We assume that our company can obtain up to $100,000 in cash at 1% per 
month. Thus each dollar of next month’s cash borrowed to meet current needs yields 


1 
— ~ 0.9901 
Lol 0.990 


dollars now. 

Similar loss arcs connect cash and bond nodes in Figure 10.23. For exam- 
ple, arc (2, 7) represents cash invested in bonds during week 2. The loss multiplier 
a2,7 = 0.998 corresponds to a 0.2% investment fee. The similar loss on arc (7, 2) 
models an identical fee to convert bonds into cash. 

The object of the management problem depicted in Figure 10.23 is to have as 
much money as possible at the end of the planning period. Thus the only nonzero 
objective function coefficient occurs on node 5 outflow. 


Gain and Loss Network Flow Models 


To reduce gain/loss flow examples such as cash management problems to standard 
network flow form, we have only to include multipliers a;; in the minimum cost 


network flow format of ; 


mand b,, and arcs (i, j) € A having capacity u;; and multiplier a;; is 


<< min y Ci jXi,j 


(pea 

s.t. = OXI — = Kaj = Dy - forallkeV 
eA (pea 
QO< Ay SU; for all G Pea 


Table 10.5 illustrates for our cash flow example. An optimal solution returns 
$489,311 at the end of the planning period. Nonzero flows are on arcs highlighted in 
Figure 10.23. 


SAMPLE EXERCISE 10. 30: FORMULATING FLows WITH GAINs | = 


Taking numbers next to nodes as net demands b,, quantities in boxes on arcs as gain 
multipliers a;;, and unboxed numbers on arcs as costs c; j, formulate the flow with 
gains model corresponding to the following figure: 


tr 2S 2 ee Rk? ee AEE ee Se pe ee SE ee 
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TABLE 10.5 Tinyco Example Model 


max 1 (max return) 


St. 0.9901 x94 + 0.9980%6 1 — X1.2- 16 = -100 (node 1) 
1.005%; 9 + 0.990123 5 + 0.9980x79-%21-%23-%27 = 300 (node 2) 


1.005x, 3 + 0.9901x43 + 0.9980x7 3 —x32—*34-%33 = 500 (node 3) 
1.005x3 4 + 0.9901x5 4+ 0.9980X9.4 —%43-X45—X4q = 10 (node 4) 
1.005x45 + 0.9980x19.5 —X5,4-7 = 30 (node 5) 
0.9980%16 -61-%67 = -200 (node 6) 
0.9980x5.7 + 1.009x67-x7.2-x78 = 0 (node 7) 
0.9980x3 ¢ + 1.009x7.g— xg 9—-Xg.9 = 0 (node 8) 
0.998049 + 1.0098 9 — X9.4-X9 10 = 0 (node 9) 
1.009X9 19 — X19,5 = 0 (node 10) 
X_1 $100 (cash limit) 
X32 $100 

X43 <100 

x54 <100 

all x; ;20 


(j= 


Modeling: Following format} 10.51 |, the model is 


min 3x12 + 14x23 + 12x31 
Stl 2X31 —- X12 =0 

2x1,2 — X23 = 0 

2X73 =a 3 


X12, X2,3,%3,1 > 0 


Tractability of Network Flows with Gains and Losses 


As with multicommodity flows, models with gain and loss flows are linear programs 

ath some exploitable structure. It is still convenient to think in terms of flows and 
represent models in simple diagrams. 

Still, many elegant properties of ordinary flows are lost when gains and losses 


sre permitted. In particular, integrality property | 10.20 | does not extend. 


with gains and losses may be fractional. 


The simple instance in Sample Exercise 10.30 provides an example. All data 
ere integer, but the only feasible solution is 
6 12 


BA ee RS) Ng 
Te hg a 


EXERCISES 


!-1 The figure that follows depicts a minimum cost net demand, while those on arcs show unit cost and 
»-twork flow problem. Numbers on the nodes show _ capacity. 
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(a) [8] Identify the node set V and the arc set A of 
the network. 

(b) [&] Classify all nodes as source, sink, or trans- 
shipment. 

(c) Verify that total supply equals total demand. 
(d) [| Formulate the corresponding minimum cost 
network flow problem as a linear program. 

(e) [| Show the node-arc incidence matrix for the 
network. 


10-2 Do Exercise 10-1 for the network 


-80 
(2, ) 


(0, 10) 
Y) 
3 


—1 Oe. 0 1 0 


(a) [Q] Explain why it is a node—arc incidence ma- 
trix. 
(b) [Q] Draw the corresponding digraph. 


10-4 Do Exercise 10-3 for the matrix 
-1 -l 0 0 0 
1 0 1 il 0 


10-5 The following digraph represents a network 
flow problem with values at nodes showing net de- 
mand. 


50 @ (3) 80 


(a) [Q] Verify that total supply exceeds total de- 
mand. 

(b) [] Add a new sink to create an equivalent net- 
work with total supply equal to total demand. 


10-6 Do Exercise 10-5 for the network 


°@ 


-100 @) ) 60 


10-7 Super Sleep is a company making matresses for 
king-size beds. Matresses can be shipped directly 
from either of its plants to retail store customers, 
or they may be transshipped through the company’s 
single warehouse. The table that follows shows the 
unit cost of shipping matresses from the plants and 
the warehouse to Super Sleep’s 2 customers. Ship- 
ping from either plant to the warehouse costs $15 
per matress. The table also shows the number of 
matresses that can be produced at each plant over 
the next week and the number demanded by each 
customer. 


Shipping Costs 

j=1 j=2 Capacity 
Plant 1 25 30 400 
Plant 2 45 23 600 
Warehouse 11 14 — 
Demand 160 700 _ 


Super Sleep seeks a minimum total shipping cost way 
to supply its customers. 


(a) [§] Formulate a linear program to determine an 
optimal shipping plan. 

(b) [| & Use class optimization software to com- 
pute an optimal solution to your LP. 

(c) [&] Show that your LP can be represented as a 
minimum cost flow model (with total supply = 


total demand) by sketching the corresponding 
digraph and labeling as in Exercise 10-1. 

d) [| Classify nodes in your digraph as source, 
sink, or transshipment. 


10-8 Crazy Crude oil company can produce 1500 
narrels per day from one of its fields and 1210 from 
the other. From there the crude oil can be piped to 
either of two tank farms, one at Axel and the other 
at Bull. Axel then trucks oil on to the Crazy Crude 
refinery at $0.40 per barrel to help meet its daily de- 
mand of 2000 barrels. Bull trucks to the refinery at 
$0.33 per barrel. It costs Crazy Crude $0.10 per bar- 
-el to pipe from field 1 to Axel and $0.35 per barrel 
to pipe from field 1 to Bull. Corresponding values 
‘or field 2 are $0.25 and $0.56. Also, the tank farms 
2an truck between themselves at $0.12 per barrel. 


a) through (d) as in Exercise 10-7. 


10-9 Determine whether each of the following se- 
guences is a chain, a path, a cycle, and/or a dicycle of 
*he network in Exercise 10-1. 


a) [6] 1-2+4-3-1 
>) 342 

ce) & 3-441 

4) [QO] 3-4-5-3 


10-10 Determine whether each of the following se- 
suences is a chain, a path, a cycle and/or a dicycle of 
se network in Exercise 10-2. 


a) 2-342 

b) 1-3-5-42-1 
) 2-3-4-5 

5 245 


19-11 The following digraph shows a partially solved 
=inimum cost network flow problem with node la- 
>els indicating net demand and arcs labels showing 
snit cost, capacity, and current flow. 


-100 


, = 
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(a) Verify that the current flow is feasible. 

(b) [S| Generate the six possible cycle directions of 
flow change. 

(c) Verify for the first (one) of your cycle directions 
that a step A in the direction would leave flow 
balanced at all nodes. 

(d) [| Determine whether each of your cycle direc- 
tions is improving. 

(ec) [] Determine whether each of your cycle direc- 
tions is feasible. 

(f) [&] For those directions that are feasible, com- 
pute the maximum step size 4 that could be ap- 
plied without losing feasibility. 


10-12 Do Exercise 10-11 for the network 
(10, 27, 5) 


-45 G@) 


(20, 40, 40) 
(1, 5, 5) 


60 @) -20 
10-13 Solve each of the following by rudimentary cy- 
cle direction Algorithm 10A starting from the solu- 
tion given in the figure. Find needed cycle directions 
by inspection. 


(a) [| The network in Exercise 10-11 
(b) The network in Exercise 10-12 


10-14 Suppose that a problem could be modeled 
with equal validity as a linear program with 1000 con- 
straints and 10,000 variables, or a minimum cost net- 
work flow with 1000 nodes and 10,000 arcs. Which 
model would be the most tractable, and why? 


10-15 Add an artificial node and artificial arcs to pre- 
pare each of the following networks for two-phase or 
big-M solution starting with a zero flow on all orig- 
inal arcs. Show the starting flow, and verify that it 
balances at the artificial node. 


(a) [9] The network of Exercise 10-1 
(b) The network of Exercise 10-2 


10-16 Each of the following depicts a minimum cost 
network flow problem, with labels on nodes indicat- 
ing net demand and those on arcs showing unit cost 
and capacity. Determine for each (without solving) 
whether any unique optimal flow would have to be 
integer valued. 
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(a) (4,5) 


(b) 


() BI 


Cc) 


(1.414, 70) 


(7.67, ) 


10-17 The Quick Chip gravel company has received 
a contract to supply two new construction projects in 
the towns of Brock and Wurst. A total of 60 truck- 
loads are needed at Brock in the next month and 90 
at Wurst. Quick Chip has idle gravel pits in the towns 
of Nova, Scova, and Tova, each with a monthly pro- 
duction capacity of 50 truckloads. Travel distances 
from each pit to each project site are shown in the 
following table. 


Pit To Brock To Wurst 


Nova 23 ET 
Scova 8 94 


Tova 53 41 


The company wants to fulfill its contract at least total 
truck travel distance. 


(a) [] Formulate a linear program to choose an op- 
timal shipping plan. 


(b) | & Use class optimization software to com- 
pute an optimal solution to your LP. 

(c) EQ] Show that your LP can be represented as 
a minimum cost transportation problem by 
sketching the corresponding digraph and label- 
ing as in Exercise 10-1. 

(a) [| Sketch the transportation tableau represen- 
tation of the problem with flows equal to your 
optimal solution in part (b). 


10-18 Maize Mills has 800 thousand, 740 thousand 
and 460 thousand bushels of corn stored at its three 
rural elevators. Its three processing plants will soon 
require 220 thousand, 1060 thousand and 720 thou- 
sand respectively, to make cornstarch. The following 
table shows the cost per thousand bushels of shipping 
from each elevator to each plant. 


Plant 
Elevator 1 2 3 
1 OE yp 


2 Sig ila alla 
3 EZ sth Sle) 


Maize wants to move its corn to plants at minimum 
total shipping cost. 


(a) through (d) as in Exercise 10-17. 


10-19 Senior design students are negotiating which 
of the four members of the team will take primar 
reponsibility for each of the four project tasks the 
team must complete. The following table shows the 
composite ratings (0 to 100) they have prepared tc 
estimate the ability of each member to manage each 
task. 


Task Rating 
Member 1 2 3 4 
it 90 78 45 69 
2 WI SA PSO REE) 
3 85° O00. 85: 193 
4 40 80 65 39 


The team wants to find a maximum total score plaz 
that allocates exactly one task to each team member 


(a) [6] Formulate a linear assignment problem (LP 
to choose an optimal plan. 

(b) [| & Use class optimization software to com 
pute an optimal assignment. 

(c) [6] Show that your assignment problem can b: 
represented as a minimum total cost flow mods 
by sketching the corresponding digraph and ls 
beling as in Exercise 10-1. 


&] A feasible assignment must have decision 
variables = Oor1 in part (a), yet the model 
can be solved as a linear program. Explain how 
the underlying network nature of the problem 
makes this possible. 


»-20 Paltry Properties has just acquired four rental 
mes. Paltry wishes to have the houses painted 
ain the next week so that all can be available for 

= prime rental season. This means that cach house 

have to be painted by a different contractor. The 

ywing table shows the bids (thousands of dollars) 
-cived from four contractors on the four houses. 


Painter Bid 
House 1 7 3 4 
1 Mey IkBie MSH alles) 
2 29 eed 5:0) 22) 
3 PLO AS P| 
4 So) leg 40h 225 


“try want to decide which bids to accept in order 
paint all houses at minimum total cost. 


through (d) as in Exercise 10-19 


+21 Formulate as a minimum cost network flow 
“9dlem, and solve by inspection, the problem of 
=ding a shortest path from the specified source to 
»= specified sink in each of the following networks. 
se costs specified on the original digraph. 


S| Source 3, sink 2 in the network of Exercise 
10-11 
Source 1, sink 3 in the network of Exercise 10-12 
(O] Source 1, sink 4 in the network of Exercise 
10-30 
Source 3, sink 2 in the network of Exercise 10-31 


™-22 Suppose that an analyst needs to compute 
»< shortest path between two specified nodes of a 
00-node 100,000-are digraph with nonnegative 
=< costs. Would shortest path methods of Chapter 
r a network flow formulation solved by a cycle 
“rection algorithm be likely to provide the most ef- 
Sent solution? Why? 
1-23 A relief agency is urgently trying to get the 
 2ximum possible quantity of supplies from its base 
Alto to the volcano-ravaged city of Epi. One 
ailable road goes via Billi. The agency estimates 
sat the Alto-to-Billi part of that road can carry 
» 9) tons per day, and the Billi-to-Epi segment, 320 
as. A second route goes via Chau and Domo, with 
y= Alto-to-Chau part having capacity 650 tons, the 
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Chau-to-Domo section, 470, and the Domo-to-Epi 
segment, 800. There is also a small mountain road 
with capacity 80 tons that connects Billi to Domo. 


(a) [§] Formulate this problem as a maximum flow 
model by sketching the associated digraph. In- 
dicate the source, the sink, and all capacities. 

(b) [QJ Solve your maximum flow problem by in- 
spection. 

(c) [Q] Show how to modify your digraph of part 
(a) to represent the model as a minimum cost 
network flow problem. 


10-24 Makers of the new Ditti Doll are urgently try- 
ing to get as many to market as possible because a 
craze has created almost unlimited demand. One 
plant can supply up to 8 thousand per week to its 
distribution center, but that center can then get only 
3 thousand per week to east region customers, and 1 
thousand to west. The other plant supplies up to 3 
thousand per week to its (distinct) distribution cen- 
ter, which can ship 2 thousand per week to the cast 
and the same number to the west. 


(a) through (c) as in Exercise 10-23. 


10-25 Formulate as a minimum cost network flow 
problem, and solve by inspection, the problem of 
finding a maximum flow from the specified source to 
the specified sink in each of the following networks. 
Use capacities specified on the original digraph. 


(a) EQ] Source 3, sink 2 in the network of Exercise 
10-11 

(b) Source 1, sink 3 in the network of Exercise 10-12 

(c) [] Source 1, sink 4 in the network of Exercise 
10-30 

(d) Source 3, sink 2 in the network of Exercise 10-31 


10-26 Return to the Super Sleep problem of Exer- 
cise 10-7, and suppose now that we wish to plan over 
a 2-week time horizon. Customer demands remains 
160 and 700 in the first week, but they are predicted 
to be 300 and 810 in the second. There is no initial 
inventory, but matresses may be held in the ware- 
house at $10 per week. All other parameters are the 
same in each week as those given in Exercise 10-7. 


(a) [| Formulate a time-expanded linear program 
to determine an optimal shipping and holding 
plan. 

(b) | & Use class optimization software to com- 
pute an optimal solution to your LP. 
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(c) [§| Show that your LP can be represented as a 
minimum cost flow model (with total supply = 
total demand) by sketching the corresponding 
digraph and labeling as in Exercise 10-1. 


10-27 Return to the Crazy Crude problem of Ex- 
ercise 10-8, and suppose now that we wish to plan 
over a 2-day time horizon. Refinery demand remains 
2000 on the first day but will be 3000 on the second. 
There are no initial inventories, but either tank farm 
can hold over petroleum at $0.05 per barrel per day. 
All other parameters are the same on each day as 
those given in Exercise 10-8. 


(a) through (c) as in Exercise 10-26. 


10-28 Demonstrate that columns of the node-arc in- 
cidence matrix corresponding to arcs in each of the 
following cycles of the digraph in Exercise 10-1 form 
a linearly dependent set. 


(a) Bl 2,5); 6,2) 

(b) (4,2), (5,2), (4, 5) 

(c) I 2), 4,2), (1, 4) 
(d) (1,4), 4,5), 6,3), 3, D 


10-29 The following depicts a network flow prob- 
lem, with labels on nodes indicating net demand and 
those on arcs showing capacity. 


For each of the following lists of possible nonbasic 
arcs, either compute the corresponding basic solu- 
tion and indicate whether it is basic feasible, or ap- 
ply principle to demonstrate that the unlisted 
arcs do not form a basis of the implied flow balance 
constraints. 


(a) © (1,2), 2, D, G4). 6. 4) 
(1, 3), (2, 5) upper-bounded 

(b) (1,2), (1,4), G. 4), (4,5), G, 4) lower-bounded, 
(1, 3), (3, 5) upper-bounded 

(c) I 4,3), @, 1), 2.5). 6. 4), 6, 4) 
bounded, (1, 4) upper-bounded 

(d) (1,2), 0, 4), (2, 3), @, 4) lower-bounded, (2,5) 
upper-bounded 


lower-bounded, 


lower- 


e) 1 d.2),4.4).4.5) lower-bounded. 
(2,5), (3, 4) upper-bounded 

(f) (,2), 2,1), 2,5), 3, 4), 6, 4) lower-bounded. 
(1, 4) upper-bounded 

(g) [8] 1.2). 0.4), GB. 4), 6.4) — lower-bounded 
(1, 3), (2, 5), (3, 5) upper-bounded 

(h) (1,2), 4, 4), @, 4), (4, 5) lower-bounded. 
(1, 3), (2, 5) upper-bounded 


10-30 The following digraph depicts a partially 
solved minimum cost flow problem with labels on 
nodes indicating net demand and those on arcs show- 
ing unit cost, capacity, and current flow. 


(a) [Q] Verify that the given solution is a basic fea- 
sible solution for basis {(1, 2), (1, 3), (2, 4)}. 

(b) [] Compute all simplex directions available 2 
this basis. 

(c) [Q] Determine whether each of the simplex di- 
rections is improving. 

(d) [&] Regardless of whether they are improving 
determine the maximum feasible step 4 that 
could be applied to each of the simplex direc- 
tions. 


10-31 Do Exercise 10-30 on the problem 


P 
(15, 15, 15) 


-60 


and basis {(1, 2), (3, 1), 6, 4}. 


10-32 Apply network simplex Algorithm 10B = 
compute an optimal flow in each of the followine 


—_— 


networks. Start from the flow given in the figure, 
using the basis specified below. 


(a) [6] The network of Exercise 10-11 with basis 
(@, 2), 2,4), G, 4)} 

(b) The network of Exercise 10-12 with basis 
GL 2a. (25,3) (45.3) 

(c) [|] The network of Exercise 10-30 with basis 
{(1, 2), (1, 3), 2, 4)} 

(d) The network of Exercise 10-31 with basis 
{7, 2), 3,1), G, 4} 


10-33 Return to the partially solved minimum cost 
network flow problem of Exercise 10-30. 


a) [§] Verify that the given flow is feasible. 

(b) [&] Construct the residual digraph correspond- 
ing to the current flow. 

c) [&] Use Floyd-Warshall Algorithm 9B on your 
residual digraph to identify an improving fea- 
sible cycle direction with respect to the given 
flow. 

d) [| Compute the maximum feasible step 1 that 
could be applied to your cycle direction. 


10-34 Do Exercise 10-33 on the network of Exercise 
10-31. 

10-35 Apply cycle canceling Algorithm 10C to com- 
pute an optimal flow in each of the following net- 
works. Start from the flow given in the figure, and 
identify negative dicycles in residual digraphs by in- 
spection. 


a) [§] The network of Exercise 10-11 
b) The network of Exercise 10-12 
c) [>] The network of Exercise 10-30 
4d) The network of Exercise 10-31 


10-36 The following digraph depicts a partially 
solved minimum cost flow problem with labels on 
the nodes indicating net demand and those on the 
arcs showing unit cost, capacity, and current flow. 


(3, 40, 25) 


25 =o) 
S S 
we 
8 S 
ice) 
& oS 
20 @) Ci, 15, 0) 40 


2) [€] Verify that the given solution is a basic fea- 
sible solution for basis {(1, 2), (1, 3), (3, 4)}. 
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(b) [9] Identify a cycle for which the corresponding 
cycle direction could be pursued by cycle cancel- 
ing Algorithm 10C but not by network simplex 
Algorithm 10B with the basis of part (a). 

(c) [Q] Identify a cycle for which the correspond- 
ing cycle direction could be pursued by network 
simplex Algorithm 10B with the basis of part (a) 
but not by cycle canceling Algorithm 10C. 


10-37 Three workstations are located on the circu- 
lar conveyor of a manufacturing facility. Most of the 
flow between them moves from one station to the 
next on the conveyor. However, 7 units per minute 
must move from each station to the station after the 
next. As much as possible of this 2-step flow should 
be carried within the 11 units per minute capacity 
of each link of the conveyor. The rest will be trans- 
ported manually. 


(a) [Q] Formulate a linear program to determine an 
optimal way to carry the flows. 

(b) [| Q Use class optimization software to com- 
pute an optimal solution to your LP. 

(c) [&] Show that your LP can be represented as a 
multicommodity flow problem by sketching the 
corresponding digraph and labeling with costs, 
capacities, and net demands. 

(da) [| Explain why your multicommodity model 
would give meaningless results if all flow were 
combined into a single commodity. 

(ce) [| The optimal solution in this problem is frac- 
tional. Explain how this is compatible with the 
network nature of your multicommodity flow 
model. 


10-38 The Wonder Waste disposal company has 5 
truckloads of nuclear waste and 5 truckloads of haz- 
ardous chemical wastes that must be moved from its 
current cleanup site to nuclear and chemical disposal 
facilities, respectively. The following table shows 
that many of the available roads are restricted for 
one or the other type of waste. 


Road 
From To Nuclear OK Chemical OK 
Site NDisp Yes No 
Site CDisp Yes Yes 
Site Inter No Yes 
NDisp CDisp No Yes 
CDisp Inter Yes No 
Inter NDisp Yes Yes 


Also, Wonder Waste wants to distribute any risk by 
allowing no more than half the 10 total truckloads on 


550 Chapter Ten e Network Flows 


any road. One particular road, the link from cross- 
ing Inter to the nuclear disposal facility, is especially 
well suited to hazardous transfer because it runs 
through very remote areas. Wonder Waste seeks a 
feasible shipping plan that maximizes the use of that 
road. 


(a) through (e) as in Exercise 10-37. 


10-39 Maine Miracle’s 2 restaurants sell lobsters ob- 
tained from 3 fisherman. A total of 350 lobsters per 
day are served at the first restaurant, and 275 at the 
second. Each fisherman can ship up to 300 per day, 
but not all arrive suitable for serving. The follow- 
ing table show the cost (including shipping) and the 
yield of servable lobsters for each combination of 
fisherman and restaurant. 


Restaurant 
Cost Yield (%) 
Fisherman 1 2 1 2 
1 $7 $7 70 60 
2 8 8 80 80 
3 5) 5 60 70 


Maine Miracle seeks a minimum total cost way to 
meet its restaurant needs. 


(a) [] Formulate a linear program to determine an 
optimal plan. 

(b) [Q] GI Use class optimization software to com- 
pute an optimal solution to your LP. 

(c) [S| Show that your LP can be represented as a 
minimum cost flow model with gains or losses 
by sketching the corresponding digraph and la- 
beling as in Sample Exercise 10.30. 

(d) [§] The optimal solution in this problem is frac- 
tional. Explain how this is compatible with the 
network nature of your flow with gains or losses 
model. 


10-40 A new grocery store has 3 weeks to train its full 
staff of 39 employees. There are 5 employees now. 
At least 2 employees must work on preparing the 
store during the next week, 5 employees the week 


after, and 10 in the final week before opening. Em- 
ployees assigned to these duties earn $300 per week. 
Any other available employees, including those who 
were themselves trained in just the preceding week, 
can be assigned to train new workers. If an employee 
trains just one other, the two of them cost $500 for the 
week. If two are trained, the three employees cost 
$800 per week, including trainer overtime. Manage- 
ments seeks a minimum total cost plan to meet all 
requirments. 


(a) through (d) as in Exercise 10-39. You may use 
nonzero lower bounds on some variables. 


10-41 Forest fire control organizations in Canada’s 
provinces® must reposition the numbers of observa- 
tion aircraft available at stations i = 1,...,11 ona 
daily basis to adjust for changing fire threats. The 
required number 7; and the present number p; are 
known for all stations, along with the cost c;; of mov- 
ing an aircraft from any station to station j. Explain 
how the problem of choosing a minimum total cost 
plan for repositioning aircraft can be modeled as a 
transportation problem by identifying the sources. 
sinks, supplies, demands, and arc costs. 


10-42 A new highway’ is being built through terrain 
points i = 1,...,40. The distance from i toi+1 
is d;. To level the route, net earth deficits b; truck- 
loads must be corrected at all nodes (6; < 0 if surplus. 
>>; b; = 0). This will be done by moving truckloads 
of earth from surplus to deficit points along the route 
of the highway, but the same earth should not be han- 
dled more than once. Explain how the problem o! 
choosing a minimum total truck travel distance level- 
ing plan can be modeled as a transportation problem 
by identifying the sources, sinks, supplies, demands. 
and arc costs. 


10-43 To estimate the impact of proposed tas 
changes, the U.S. Department of the Treasury® main- 
tains two data files of records statistically charac- 
terizing the taxpayer population. Each of the i = 
1,..., 10,000 records in the first file represents < 


known number of families a; and describes corre- 
sponding characteristics such as family size and age 


Based on P. Kourtz (1984), “A Network Approach to Least-Cost Daily Transfers of Forest Fire 


Control Resources,” INFOR, 22, 283-290. 


7Based on A. M. Farley (1980), “Levelling Terrain Trees: A Transshipment Problem,” /nformation 


Procession Letters, 10, 189-192. 


Based on F. Glover and D. Klingman (1977), “Network Application in Industry and Government,” 


AIIE Transactions, 9, 363-376. 


distribution. Records j = 1,..., 40,000 in the sec- 
ond file also represent a known number of families 
5; and contain some of the same characteristics as the 
first (b';4; = )0; bj). However, most of the entries 
in the second file relate to the sources of income for 
family class j. To do a better job of analyzing pro- 
posals, Treasury wants to merge these files into one 
with new records containing information drawn from 
doth inputs. Each new record will represent a col- 
lection of families formed by matching some or all 
of those in population a; with some of all in b;. The 
guality of the similarity between classes i of the first 
fle andj of the second can be described by a distance 
measure d;;, and the Treasury seeks a minimum to- 
tal distance merge. Explain how this problem can 
de modeled as a transportation problem by identify 
sources, sinks, supplies, demands, and costs. Also, 
explain how an optimal flow can be understood as a 
merge. 


10-44 Freight trains’ run a regular weekly schedule 
in both the forward and reverse directions of a rail- 
road’s main line through section boundary points 
/=1,...,22. Dividing the week into hourly time 
locks t = 1,..., 168 (with ¢ = 1 following t = 168), 
a train leaving i bound for j > i advances through 
doth time and space as it transits sections (i,i +.1), 
i+1,i+ 2), and so on. Summing requirements 
for all scheduled trains to pull the anticipated load 
over grades in the section, engine needs f;,; can be 
estimated for each forward section (i, i+1), and each 
ume ¢ to ¢ + p;, where p; is the number of hours re- 
guired to transit the section. Reverse requirements 
©, provide the same information for trains moving 
satrtoi—1att+q; with q; the time to transit seg- 
ment (7,i — 1). We assume that all engine units are 
identical. Those located at any place and time where 
they are not immediately needed, may be held there, 
turned around to go the opposite way, or added as 
extras on a passing train. Cost c;, d;, and h reflect 
the cost of running one unit over segment (7,7 + 1), 
running one over (i,7 — 1), and holding for an hour 
ai any site, respectively. Show that the problem of 
computing a minimum total cost engine schedule 
can be modeled as a network flow problem over 
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a suitable time-expanded network, by sketching a 
representative node and showing all adjacent nodes, 
arcs, costs, bounds, and node net demands. Some 
arcs will have nonzero lower bounds. 


10-45 A substantial part of United Parcel Service’s'” 
freight traffic moves as trailer-on-flatcars (i.e., with 
truck trailers traveling most of the way on railroad 
flatcars). The required number of truckloads dj; to 
be shipped between points i,j =1,..., in this way 
is known, but UPS can use either its own trailers at 
unit cost c;; or rent trailers from the railroad at unit 
cost r;;. Rented trailers can be left anywhere, but 
UPS wishes to balance the number of its own trail- 
ers available at every point. That is, the number of 
company trailers inbound at any point should equal 
the number outbound. If necessary, trailers may be 
returned empty from i to j at unit cost e,; to meet 
this requirement. Show that the problem of finding 
a minimum total cost shipping plan can be modeled 
as a (single-commodity) network flow problem by 
detailing the arcs that would join each node i to an- 
other j, including both their cost and any applicable 
capacity. Also indicate the net flow demand at each 
node. (Hint: Represent rented trailer flows indi- 
rectly from flows of loaded company-owned trailers 
with capacity dj.) 

10-46 KS brand tires are shaped in changeable 
molds type i = 1,..., m, installed in the company’s 
40 presses. Production plans dictate the minimum 
r,, and maximum 7;, numbers of molds 7 that should 
be operational during time period t = 1,...,, and 
the planning period begins t = 0 with numbers of 
molds b; installed (}°; b; = 40). There is an adequate 
supply of molds, but changeovers from one mold to 
another are expensive, costing an amount c; depend- 
ing on the mold installed. All 40 presses should be 
in use in each time period. Demonstrate that the 
problem of finding a minimum total changeover cost 
mold schedule can be modeled as a network flow on 
a suitable time-expanded network by sketching the 
graph for a case with m = 2 and n = 3, and show- 
ing all nodes, arcs, costs, bounds, and node net de- 
mands. Some arcs will have nonzero lower bounds. 
(Hint: Also include supernodes balancing the total 


Based on M. Florian, G. Bushell, J. Ferland, G. Guerin, and L. Nastanshy (1976), “The Engine 
Scheduling Problem in a Railway Network,” JNFOR, 14, 121-138. 
\0Based on R. B. Dial (1994), “Minimizing Trailer-on-Flat-Car Costs: A Network Optimization 


Model,” Transportation Science, 28, 24-35. 


Based on R. R. Love and R. R. Vemuganti (1978), “The Single-Plant Mold Allocation Problem 
with Capacity and Changeover Restrictions,” Operations Research, 26, 159-165. 
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number of molds removed and inserted in each time 
period.) 

10-47 Major league baseball umpires’? work in crews 
that move among league cities to officiate series of 2— 
4 games. After every series, all crews move on to an- 
other that must involve different teams. To provide 
adequate travel time, any crew that works a series 
ending with a night game must also not go directly 
to one starting with a day game. Within these limits, 
league management would like to plan crew rotation 
to minimize the total of city i-to-city j travel costs c;,;. 
Experience has shown that good results can be ob- 
tained by deciding each all crew move independently 
(i.e., without regard to where crews were before the 
most recent series or where they will be after the 
next), Explain how the problem of planning a move 
can be modeled as a linear assignment problem by 
describing the two sets being matched, the collection 
of feasible pairings, the associated linear costs, and 
whether the total is to be minimized or maximized. 


10-48 One of the ways that airlines operating 
through major hubs can improve their service is to al- 
low as many transferring passengers as possible who 
land at one of the scheduled arrival-departure peaks 
to carry on with their next flight on the same plane.’ 
Such through-flight connections must involve flights 
scheduled for the same type of aircraft, and flights 
with sufficient arrival to departure time to complete 
the required servicing. The number of passengers p;,; 
arriving on flight i and continuing on flight j at any 
peak can be estimated in advance. Explain how this 
problem of optimizing through flights can be mod- 
eled as a linear assignment problem by describing 
the two sets being matched, the collection of feasi- 
ble pairings, the associated linear costs, and whether 
the total is to be minimized or maximized. 


10-49 As commercial airliner'* makes stops j = 
1,...,n of its daily routine and returns to where it 
started it takes on fuel for the next leg. Fuel is added 
at stop j to assure that the plane will arrive at stop 
j +1 with at least the required safety reserve 71. 
Fuel unit costs c; (dollars per pound) vary consider- 
ably from stop to stop, so it is sometimes economical 


to carry more fuel than the minimum required in or- 
der to buy less at high-cost stops. However, the take- 
off fuel load at any j must not exceed safety limit #. 
The amount of fuel at takeoff also affects the weight 
of the aircraft and thus its fuel consumption during 
flight. For each leg from stop j to j + 1, the fuel re- 
quired can be estimated as a constant @; plus a slope 
f; times the onboard fuel at takeoff from j. 


(a) Formulate this fuel management problem as a 
linear program in the decision variables (j = 
ib aay) 


x; & fuel added at stop j 
y; & onboard fuel at takeoff from j 


Assume that stop 1 is the successor of stop 7. 
and use nonzero lower bounds if needed. 

(b) Show how your model can be viewed as a flow 
with gains where the x; correspond to 1-ende¢ 
arcs, and the arcs associated with the y,; have 
both upper and (nonzero) lower bounds. Sketch 
the graph for a case with n = 3 and show all 
nodes, arcs, costs, bounds, and node net de- 
mands. 


10-50 American Olean makes product families 
i=1,...,10 of tile at plants p = 1,...,4to meet de- 
mands d;, (square feet) at sales distribution points 
(SDPs) k = 1,...,120. Variable costs of produc- 
tion and transportation total c;,,. per square foot te 
make tile of family i at plant p and ship to SDP & 
Each plant can produce up to 100% of capacity, with 
u;» being the capacity if plant p makes only group 
i. Management wants to find a minimum total cost 
way to meet demand. 


(a) Formulate an LP model to compute an op- 
timal plan in terms of the decision variables 
G21 S10 4k S120) 


Xip,x & fraction of capacity at plant p devoted 
to making tile family i for shipment 
to SDP k 


12Based on J. R. Evans, (1988), “A Microcomputer-Based Decision Support System for Scheduling 
Umpires in the American Baseball League,” Interfaces, 18:6, 42-51. 
13Based on J. E Bard and I. G. Cunningham (1987), “Improving Through-Flight Schedules,” JE 


Transactions, 19, 242-250. 


14Based on J. S. Stroup and R. D. Wollmer (1992), “A Fuel Management Model for the Airline 


Industry,” Operations Research, 40, 229-237. 


15Based on M. J. Liberatore and T. Miller (1985), “A Hierarchial Production Planning System,” 


Interfaces 15:4, 1-11. 


5) Show that your model can be viewed as a flow 
with gains of the transportation problem type 
by identifying sources, sinks, supplies, demands, 
arc gain multipliers, and arc costs. 


10-51 The figure that follows shows a part of a distri- 
Sution network like those used by Shell Oil Com- 
cany!° to supply the midwest with its three main 
classes of product: gasoline, kerosene/jet fuel, and 
‘uel oil. 
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Solid links shown indicate available pipelines, 
but products can also be shipped by barge from the 
refineries at Houston and Norco to depots at Wood 
River and/or Coraopolis. Wood River is also a re- 
finery. Flows must meet known demands d,,, for the 
various products p at all nodes i. The three refineries 
have known production capacities b;, for production 
of product p at refinery i, and a pipeline from point 
i to point j can carry combined total of at most u;; 
barrels of product. Barge capacity is essentially un- 
limited. Refining costs are assumed fixed, and cost 
per barrel c;; to transport along links from 7 to j is 
the same for all products. 


(a) Formulate an LP model to determine a mini- 
mum total cost distribution plan. 

(b) Show that your model can be viewed as a multi- 
commodity flow problem by sketching the cor- 
responding network, labeling arcs with costs 
and capacities, and nodes with net demands. 


Bazaraa, Mokhtar S., John J. Jarvis, and Hanif Sher- 
ali (1990), Linear Programming and Network Flows, 
Wiley, New York. 


*°Based on T. K. Zierer, W. A. Mitchell, and T. R. White (1976), “Practical Applications of Linear 
~~ >eramming to Shell’s Distribution Problems,” Interfaces, 6:4, 13-26. 
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Discrete 
Optimization Models 


“lost network flow, shortest path, and dynamic programming models of Chapters 9 

snd 10 solve elegantly, even if decision variables are treated as discrete. Fortunately, 

such special discrete models do occur in operations research practice. Unfortunately, 

‘hey are hardly the norm. The overwhelming majority of integer and combinatorial 
ptimization models prove much more challenging. 

Before exploring methods for dealing with hard discrete models, we need to 
set some concept of their enormous range. In this chapter we formulate a series of 
“assic types using real application contexts often drawn from published reports. In 

hapter 12 we address integer programming methods. 


| 1,1 LUMPY LINEAR PROGRAMS AND FIXED CHARGES 


Jne broad class of discrete optimization problems add either/or side constraints or 
djective functions to what is otherwise a linear program. For want of a better term, 
= might call such models lumpy linear programs. 


Swedish Steel Example with All-or-Nothing Constraints 


“ne Swedish steel blending example of Section 4.2 provides an illustration. The 
model repeated below, which was formulated there, chooses a minimum cost mix of 
“rap metal and pure additives to produce a “charge” of steel. Constraints restrict 


2 chemical content of the charge. 


553 


Chapter Eleven e Discrete Optimization Models 


min 16x; + 10x. + 8x3 + 9x4 + 48x5 + 60x6 + 53x7 (cost) 
st. Xy bX. +43 4X4 +5 + X6 +7 1000 (weight) 


0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 = 6.5 (carbon) 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 < 7.5 

0.180x; + 0.032x2 + 1.0x5 630) (nickel) 
0.180x1 + 0.032x2 + 1.0x5 < 35 

0.120x; + 0.011x2 + 1.0x6 > 10 (chromium) (11.1 
0.120x1 + 0.011x2 + 1.0x6 1D) 

0.001x2 + 1.0x7 Sell (molybdenum) 
0.001x2 + 1.0x7 < 13 

ty =< 79 (availability) 
xX < 250 

De eee o7ees4() 


In real application, steel blending often has an added complexity. Some of the 
scrap elements considered in the blend may be large blocks of reclaimed steel that 
cannot be subdivided. Either the entire block is used, or none of it is. 


ILP Modeling of All-or-Nothing Requirements 


Such indivisible input elements illustrate the need to model all-or-nothing phenom- 
ena. The usual approach to dealing with such cases is to rescale to new discrete 
variables. 


can be modeled by substituting x= ujy;, with new discrete variable 


The new y; can be interpreted as the fraction of limit u; chosen. 


Swedish Steel Model with All-or-Nothing Constraints 


Suppose in our example model (11.1) that the first two ingredients had this lump) 
character. That is, we may use either none or all 75 kilograms of ingredient 1 and none 
or all 250 kilograms of ingredient 2. Then, instead of continuous decision variables 
x1 and x to reflect the quantities of each ingredient used, we simply employ discre™ 
alternatives 


A 


; | ik if scrap j is part of the blend 
i= 


0) otherwise 


In terms of these new variables, the quantity of scraps 1 and 2 included in the 
mix are 75y; and 250y2. Substituting produces the following integer linear program 


11.1 Lumpy Linear Programs and Fixed Charges Do”. 


definition H): 

min 16(75)y; + 10(250)y2 + 8x3 + 9x4 + 48x5 + 60x65 + 53x7 

st. 75yy + 250y2 + x3 x4 +x5 + 2X6 +27 = 1000 
0.0080(75)y; + 0.0070(250)y2 + 0.0085x3 + 0.0040x4 > 6.5 
0.0080(75)y1 + 0.0070(250) y2 + 0.0085x3 + 0.0040x4 < 7.5 
0.180(75)y; + 0.032(250)y2 + 1.0x5 30 
0.180(75)y, + 0.032(250)y> + 1.0xs 
0.120(75)y, + 0.011(250)y2 + 1.0% 
0.120(75)y, + 0.011(250)y> + 1.0x¢ 
0.001 (250)y» + 1.0x7 
0.001 (250)yo + 1.0x7 


5p ooo ky 2. OU 


(11.2) 


Ay IM SIAN VA IV AS IM 
Go 
rn 


y1,¥2 =Oorl 
An optimal solution now sets 
y=L =. x3 = 736.44, xj = 160.06 
Kee — G05 4 — 00 el 00 


Total cost increases to 9967.1 kroner, versus 9953.7 for the linear programming ver- 
son (11.1), because of the all-or-nothing requirements. 


SAMPLE EXERCISE 11.1: MODELING ALL-OR-NOTHING VARIABLES 


-onsider the linear program 


max 18x; + 3x2 + 9x3 

s.t. 2x1 +X2 + 7x3 < 150 
0 <x, < 60 
0 < x2 < 30 
0 < x3 < 20 


Revise the model so that each variable can be used only at zero or its upper bound. 


Modeling: Following principle , we introduce new 0-1 variables 
y; A fraction of upper bound u; used 


Then substituting, the model becomes 


max 18(60y;) + 3(30y2) + 9(20y3) 
st.  2(60y1) + (3O0y2) + 7(20y3) < 150 
y1,¥2,¥3 =O0orl 


max 1080y; + 90y2 + 180y3 
st. 120y; + 30y2 + 140y3 < 150 
y1,Y2,¥3 =Oorl 
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ILP Modeling of Fixed Charges 


Another common source of lumpy phenomena in what are otherwise linear programs 
arises when the objective function involves fixed charges. For example, a nonnegative 
decision variable x may have cost 


face ifx > 0 


A(x) A 
he 0 otherwise 


That is, a fixed initial, or construction, or setup cost, f, must be paid before continuous 
decision variable x can be used at any nonzero level. Thereafter the usual variable 
cost c of linear programming applies. 

If such fixed charges are nonnegative, which is almost always true, they can be 
modeled in mixed-integer linear programs by using new fixed charge variables. 


11.2 | Minimize obj ective functions with nonnegative fixed charges for mak ny 

variables xj > 0 can be modeled by introducing new w fixed charge variables _ 
ty 

2 = 1-0 otherwise 


The objective coefficient S y; is the fixed cost of x;, and the coefficient of 2 is 
its variable cost. 


New switching constraints are also required to link y; with corresponding x. 


= Switching constraints model the requirement that continuous variable 
2 . can be used only if a corresponding eS) variable y; = 1 by — 
- = uj y; 


where uj isa given or derived upper bound on the value of x; in any enue 
solution. : 


If y; = 1, x; can assume any LP-feasible value. If y; = 0, then x; = 0 too. 


Swedish Steel Example with Fixed Charges 


To illustrate the modeling of fixed charges, return again to our original Swedish Ste« 
model (11.1). This time, suppose that there are setup costs. Specifically, assume the 
ingredients 1 to 4 can be used in the furnace only after injection mechanisms ar 
setup at a cost of 350 kroner each. 

To model these fixed charges, we introduce new discrete variables 


i 1 if setup for j is performed 
; 0 otherwise 


fons alee 5A. 
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We also require upper bounds on feasible values of the first four xj. Bounds wu; = 
7S and uz = 250 are given in model statement (11.1). We must derive corresponding 


upper bounds for x3 and x4 in switching constraints | 11.3}. Any value implied by 
constraints on these variables is valid, although we will see in Section 12.3 that it 
helps to employ the smallest possible. For simplicity, we look here only at the first 
main constraint of model (11.1). Since it sets the total weight of the charge at 1000, 
we know that u3 = us = 1000 are valid upper bounds. 

Introducing these new variables and switching constraints of definition 
oroduces the fixed-charge version of our Swedish Steel model: 


min 16x, + 10x. + 8x3 + 9x4 + 48x5 + 60x65 + 53x7 
+ 350y; + 350y2 + 350y3 + 3504 


St. XX. +43 4+%44+K5 +6 4X7 = 1000 
0.0080x; + 0.0070x2 + 0.0085x3 + 0.0040x4 > 6.5 
0.0080x1 + 0.0070x2 + 0.0085x3 + 0.0040x4 < 7.5 
0.180x; + 0.032x2 + 1.0x5 > 30 
0.180x; + 0.032x2 + 1.0x5 S35) 
0.120x1 + 0.011%. + 1.0x¢ > 10 
0.120x, + 0.01 1x2 + 1.0x¢6 <2 
0.001x2 + 1.0x7 > 11 
0.001x2 + 1.0x7 = 13 
x, < 75y1 
x2 < 250y2 
x3 < 1000y3 
x4 < 1000y4 
X70 


Vina ya O-0n L 
An optimal solution is 
X= 79; x, =(0, x3 = 736.44, xj = 160.06 
xe = 16.5, x6 =1.00, x7 = 11.00 
Vie =O | Wael y= 


which sets up for ingredients 1, 3, and 4. Total cost is 11,017.1 kroner, versus 9953.7 
a the linear version (11.1). 


SAMPLE EXERCISE 11.2: MODEL 
-onsider a fixed-charge objective function 


min 6) (x1) + 62(x2) 
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where 
150 + 7x1 ifx, > 0 
7) A 
14) = | 0 otherwise 
and 
110 + 9x ifx. > 0 
A(x») 4 | : iim 
0 otherwise 


For each of the following systems of constraints on x; and x2, form a corresponding 
mixed-integer linear programming model. 


(a) x7 ep 28 
0<x, <3 
0<x <8 


(b) x1 +x. >8 
2x, + x2 < 10 
X1,X2 > 0 


Modeling: We introduce y; to carry fixed charges as in and switching con- 


straints as in : 


(a) In this case upper bounds 3 and 8 are provided. Thus the mixed-integer formu- 
lation is 
min 7x; + 9x2 + 150y; + 110y2 
st. x, +x >8 
x = 3y1 
X2 < 8y2 
Mien 0 
y1,¥2 =O0orl 


(b) Upper bounds on x; and x are not explicit in these constraints. Still, we may infer 
from the second main constraint that any feasible solution has x; < 5 and x2 < lt 
Thus a mixed-integer formulation is 


min 7x; + 9x2 + 150y; + 110y2 
st. xy +x. > 8 

2x, +x2 < 10 

x1 <5y1 

XD SS 10y2 

X1,x2 = 0 

y1, 2 =Oorl 


11.2 KNAPSACK AND CAPITAL BUDGETING MODELS 


In contrast to the cases of Section 11.1, which involve linear programs with some 
discrete side conditions, knapsack and capital budgeting problems are complete!) 
discrete. We must select an optimal collection of objects, or features, or projects, oF 
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investments subject to limits on budget resources. Each element is either all in or all 
out of the result, with no partial selections allowed. 


Knapsack Problems 


The simplest of these object choosing discrete problems, and indeed the simplest of 
all integer linear programs, are knapsack problems. 


constraint. - 


All knapsack decision variables are 0-1 in most applications. 

The knapsack name derives from the problem confronted by a hiker packing 
a backpack. He or she must choose the most valuable collection of items to take 
subject to a volume or weight limit on the size of the pack. 


EXAMPLE 11.1; INDY CAR KNAPSACK 


We may illustrate more realistic forms of knapsack problems by considering the (fic- 
titious) dilemma of mechanics in the Indy Car racing team. Six different features 
might still be added to this year’s car to improve its top speed. Table 11.1 lists their 
estimated costs and speed enhancements, 


TABLE 11.1 Indy Car Example Alternatives 


Proposed Feature, j 
bi dibs. lak Dimi, baie RIT) 
Cost ($ 000’s) 10:2 GON 223.05) dik ~ 98) 3ik6 
Speed increase (mph) 8 3 15 if 10 12 


Suppose first that Indy Car wants to maximize the performance gain without 
exceeding a budget of $35,000. Using decision variables 


1 if feature j is added 
4A (11.3) 
0 otherwise 
we can formulate the problem as the knapsack model 
max 8x; + 3x2 + 15x3 + 7x4+10x%5+12x%6  - (mph gain) 
at. 10.2x, + 6.0x2 + 23.0x3 + 11.1x4 + 9.8x5 + 31.6x%6 < 35 (budget) (11.4) 
Mire. ke = Oron t 


That is, we maximize total performance subject to a budget constraint. An 
»ptimal solution chooses features 1, 4, and 5 for a gain of 25 miles per hour. 

Suppose now that the Indy Car team decides they simply must increase speed by 

_ >) miles per hour to have any chance of winning the next race. Ignoring the budget, 

“ey wish to find the minimum cost way to achieve at least that much performance. 
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This scenario leads to an alternative, minimize knapsack form. With variables (11.3). 


we obtain 
min 10.2x; + 6.0x2 + 23.0x3 + 11.1x4 + 9.8x5 + 31.6x5 (cost) 
st. 8x, + 3x2 + 15x3 + 7x4 + 10x5 + 12x56 > 30 (mph required) (11.5 
Mises ke 0 ond 


This model minimizes cost subject to a performance requirement. An optimal solu- 
tion now chooses features 1, 3, and 5 at cost $43,000. 


SAMPLE EXERCISE 113: ‘FoRMUL. TING KNAPSACK MODELS — 


Readily available U.S. coins are denominated 1, 5, 10, and 25 cents. Formulate 2 
knapsack model to minimize the number of coins needed to provide change amount 
q cents. 


Modeling: We employ decision variables xj, xs, x10, and x25 to represent the number 
of coins chosen from each denomination. Then the knapsack model is 


min xy +xX5 + x40 + X25 (total coins) 
st. x1 + 5x5 + 10x19 + 25x25 = g (correct change) 
X1,%5,X19, X25 => and integer 


Notice that these discrete variables are not limited to 0 and 1. 


Capital Budgeting Models 


The typical maximize form of a knapsack problem has its single main constraint 
enforcing a budget. When there are budget limits over more than one time period 
or multiple limited resources, we obtain more general capital budgeting or multi- 
dimensional knapsack models. 


EXAMPLE 11.2: NASA CAPITAL BUDGETING 


The U.S. space agency, NASA, must dea! constantly with such decision problems im 
choosing how to divide its limited budgets among many competing missions pro- 
posed.! Table 11.2 shows a fictitious list of alternatives. 

We must decide which of the 14 indicated missions to include in program plans 
for the 2000-2024 era. Thus it should be clear that the needed decision variables are 


a | 1 if mission j is selected 


x= (11.6 


0 otherwise 


1Based on Gerald W. Evans and Robert Fairbairn (1989), “Selection and Scheduling of Advanc== 
Missions for NASA Using 0-1 Integer Linear Programming,” Journal of the Operational Research Societ 
40, 971-981. 
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TaBLE 11.2 Proposed Missions in NASA Example 


Budget Requirements ($ billion) 


2000- 2005- 2010- 2015- 2020- Not Depends 
Mission 2004 2009 2014 = 2019 2024 Value With On 
Communications satellite 6 200 — — 
Orbital microwave 2 3 —_— — — 3 — — 
Io lander 3 5 — — — 20 — — 
* Uranus orbiter 2020 10 50 > 3 
Uranus orbiter 2010 _— 5) 8 — — 70 4 3 
Mercury probe _ — 1 8 4 20 —_— 3 
Saturn probe 1 8 — — — 5 — 3 
Infrared imaging — —- — 5 — 10 11 — 
Ground-based SETI 4 5 — _ —: 200 14 — 
Large orbital structures — 8 4 — — 150 — — 
Color imaging a — 2 7 — 18 8 2 
Medical technology 5 a — — — 8 — — 
Polar orbital platform _ 1 4 1 1 300 — — 
Geosynchronous SETI _ 4 5 3 3 185 9 — 
Budget 10 12 14 14 14 
Budget Constraints 


The budget constraints that give capital budgeting problems their name limit project 
=xpenditures in particular time periods. 


11.6 i 


Budget constraints limit the total funds or other resources consumed by 
d projects, investments, and so on, in each time period not to exceed the 
amount available. 


Budget requirements in Table 11.2 span five time periods: 2000-2004, 2005- 
209, 2010-2014, 2015-2019, and 2020-2024. We form budget constraints for each of 
hese periods by summing project decision variables times their needs. 


6x1 + 2x2 + 3x3 + 1x7 +4%9+5x12 < 10 (2000-2004) 


3x2 + 5x3 + 5x5 + 8x7 + 5x9 + 8x49 (2005-2009) 
+ 7x12 + 1x13 + 4444 < 12 

8x5 + 1x6 + 4x19 + 2411, + 4013 + 5x14 < 14 (2010-2014) 

8x6 + Sxg + 7x11 + 1x13 + 3x14 < 14 (2015-2019) 

10x4 + 4x6 + 1x13 + 3x14 <14 (2020-2024) 


SAMPLE EXERCISE 11.4: FORMULATING BUDGET CONSTRAINTS 

4 department store is considering 4 possible expansions into presently unoccupied 

space in a shopping mall. The following table shows how much (in millions of dollars) 

ach expansion would cost in the next two fiscal years, and the required floor space 
a thousands of square feet). 
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Expansion, j 
1 2 3 4 


Nir a Sy Sy tes Ee a |e) 
Year2 ey 3.5ttekS) =<6:00— 422; 


Space. 2:2" 97> 5.3" 18:6 


Using decision variables 


oe! expansion j is selected 
= : 

par ls) otherwise 
formulate implied constraints on investment funds and floor space assuming that 11 
million dollars are available in each of the two years and that the expansion cannot 


exceed 17 thousand square feet. 


Modeling: The three required budget constraints are 


1.5x1 + 5.0x2 + 7.3x3 +1.9x4 < 10 (year 1 budget) 
3.5x, + 1.8x2 + 6.0x3 + 4.2x4 < 10 (year 2 budget) 
2.2X, + 9.1%. +5.3x3 + 8.6x4 < 17 (floor space) 


Modeling Mutually Exclusive Choices 


Capital budgeting problems often come with other constraints besides simple budget 
limits. For example, two or more proposed projects may be mutually exclusive. That 
is, at most one of them can be included in a solution. 

Table 11.2 indicates three such conflicts. Possibilities 7 = 4 and 5 represent 
alternative timing for the same mission. Technologies for missions j = 8 and 11 are 
incompatible. Numbers j = 9 and 14 involve two different ways to accomplish the 
SETI program. 

Such incompatibilities are easily modeled with 0-1 decision variables (11.6). 


For our NASA example, the result is 


x4tx5 <1 
Xgtxy <1 
Xo + X14 <1 


LLY EXCLUSIVE CONSTRAINTS 


Suppose ek: a feate estate Loe company is considering 5 investment deci- 
sions. Only 1 of the first 3 can be chosen because all require the same piece of land 
Investment 4 is a new office building, and investment 5 is the same project delayed 2 
year. Formulate suitable mutually exclusive constraints in terms of decision variables 


yoalael if investment j is selected 
2S 
= 10 otherwise 


i ee a Ee Se eT ee ee ee 
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Modeling: The fact that only 1 of the first 3 can be selected produces constraint 
Xp tx. +43 <1 
Since x4 and xs refer to the same investment, we also want 


Xq+x5 <1 


Modeling Dependencies between Projects 


Another characteristic relationship between projects arises when one project de- 
pends on another. We cannot choose such a dependent project unless we also include 
‘he option on which it depends. 

We can model dependencies among projects as easily as mutual exclusiveness. 


11. 8 | Dependence of cists J on choice | i can be enforced on corresponding 


variable x; cannot = 1 unless x; does too. 

Table 11.2 shows that mission j = 11 depends on mission 2 in our NASA 
example, and also that mission 7 = 3 must be chosen if any of projects 4,..., 7 is. 
implied constraints are 


Xu S X2 
x4 S X3 
x5 S X83 
Xp <3 
x7. = x3 


s AMPLE XERCISE 11.6: MODELING PROJECT DEPENDENCIES 


Phase 1 of a new city hall project will construct the first story of a building that could 
row to two stories in phase 2, and to three in phase 3. Using the variables 


| il if phase j is constructed 
Xj = 


0 otherwise 


stite corresponding dependency constraints. 


Modeling: Obviously, the second floor cannot be built without the first, and the third 
sithout the second. Thus we have depend project constraints 


x2 5% 
X35 X2 


NASA Example Model 


To complete formulation of our version of NASA’s decision problem, we need an 
»bjective function. Like almost all public agencies, NASA has many. They may try 
> maximize the intellectual gains of selected missions, maximize the direct benefit 
»o life on earth, and so on. 

We will assume that a weighted sum of these different objective functions 
zan be used to estimate a value for each mission. Resulting values are included in 
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Table 11.2. Combining with previous elements, we obtain the NASA example mode! 


max 200x; + 3x2 + 20x3 + 50x4 + 70x5 (total value) 
+ 20x65 + 5x7 + 10xg + 200x9 + 150x110 
+ 18x11 + 8x12 + 300x713 + 185x144 
st. 6x4 + 2x2 + 3x3 + Lez + 4x9 + 5x12 10 (2000-2004) 
3x2 + 5x3 + 5x5 + 8x7 + 5x9 + 8x10 (2005-2009) 


IA 


+ 7x12 + 1x13 + 4X14 < 12 
8x5 + 1x6 + 4x10 + 2x11 + 4013 + Sea < 14 (2010-2014) 
8x6 + 5xg + 7x1, + 1x13 + 3x14 < 14 (2015-2019) 
10x4 + 4x6 + 1x3 + 3x14 < 14 (2020-2024) (11 a 
Sexe <1 (mutually 3 
Pree exclusive) 
Xo xia = I 
Xu S X2 - (dependent 
X4 < x3 missions) 
X5 3X3 
X6 5X3 
X7 S X3 


x = 0 07.1... for all Sac 2-5 14 


11.3 SET PACKING, COVERING, AND PARTITIONING MODELS _ 


Our capital budget models of Section 11.2 included mutual exclusiveness constraints 
involving subsets of decision variables, at most one of which can take part in a solu 
tion. Set packing, covering, and partitioning models feature such constraints. Using 
decision variables that = 1 if a object is part of a solution and = 0 otherwise, thes: 
models formulate problems where the core issue is membership in specified subsets 


EXAMPLE 11.3: EMS LOCATION PLANNING 


As always, it will help to think of a specific example. A classic occurred when Austin 
Texas undertook a study of the positioning of its emergency medical service (EMS 
vehicles.” That city was divided into service districts needing EMS services, and ve- 
hicle stations selected from a list of alternatives so that as much of the population a 
possible would experience a quick response to calls for help. 

Figure 11.1 shows the fictitious map we will assume for our numerical version 
Our city is divided into 20 service districts that we wish to serve from some comb 
nation of the 10 indicated possibilities for EMS stations. Each station can provide 
service to all adjacent districts. For example, station 2 could service districts 1, 2. © 
and 7. Main decision variables are 


j= 


Re) he if location j is selected 
0 otherwise 


2Based on David J. Eaton, Mark S. Daskin, Dennis Simmons, Bill Bulloch, and Glen Jansma (1985 
“Determining Emergency Medical Service Vehicle Deployment in Austin, Texas,” Interfaces, 15:1, 96-11% 
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FIGURE 11.1 Service District and Candidate Locations for EMS Example 


Set Packing, Covering, and Partitioning Constraints 


The defining constraints of set packing, covering, and partitioning models deal with 
subcollections of problem objects. In our EMS example the subcollections are loca- 
tons that can provide satisfactory response for a service district. For instance, Figure 
/1.1 shows that any of locations 4, 5, and 6 can protect downtown district 12. 

Covering constraints demand that at least one member of each subcollection 
»elongs to asolution, packing constraints allow at most one member, and partitioning 
onstraints require exactly one member. Mathematical forms follow easily with 0-1 

ariables. 


subcollec- 
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i111 | Set partitioning constraints requiring that exactly one SS of = : 
collection: 7 belongs toa solution are express ac 


In our EMS example, we are covering, because each district should be protected 
by at least one location, but more is even better. In other settings, such as radio station 
location, we pack; at most one station on any particular frequency should reach a 
service area (see Exercise 11-7). The mutual exclusiveness constraints of capital 
budgeting also have the packing form. Partitioning applies when exactly one decision 
option can serve each sector. This is the case, for example, if decision variables relate 
to possible election districts, and constraints demand that each geographic unit belong 
to exactly one district (see Exercise 11-12). 


SAMPLE EXERCISE 11.7: FORMULATING SET PACKING, eee 
AND PARTITIONING S 


A university is acquiring mathematical programming software for use in operations 
research classes. The four codes available and the types of optimization algorithms 
they provide are indicated by x’s in the following table. 


Algorithm eS 
Type 1 2 3 4 
LP x x Bg x 
IP en oes 
NLP —- — x x 

Objective 3 4 6 «614 


(a) Taking objective function coefficients as code costs, formulate a set covering 
model to acquire a minimum cost collection of codes providing LP, IP, and NLP 
capability. 


(b) Taking objective function coefficients as code costs, formulate a set partitioning 
model to acquire a minimum cost collection of codes with exactly one providing LP. 
one providing IP, and one providing NLP. 


(c) Taking objective function coefficients as indications of code quality, formulate a 
set packing model to acquire a maximum quality collection of codes with at most one 
providing LP, at most one providing IP, and at most one providing NLP. 


Modeling: In each case we use the decision variables 


A 
lee 


1 if code j is chosen 
0 otherwise 
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(a) Following form , the required model is 


min 
s.t. 


3x, + 4x + 6x3 + 1444 
Xp +X. +%34+%4> 1 


X2 +X4 Sse 
x3 4+X4 Zed 
Ki ose Ororck 


(b) Following form} 11.11 |, the required model is 


min 
Sau 


3x1 + 4x2 + 6x3 + 14x4 
xy t%2+x3+%4=1 
X2 + X4 =| 
X34+X4 = 
Cea = orl 


(c) Following form| 11.10 |, the required model is 


max 
s.t. 


3x1, + 4x2 + 6x3 + 14%4 
x $x. + x3 4+%4 <1 
X2 + X4 <1 
x3 + X4 <1 
Miee eo = 0 orl 


Minimum Cover EMS Model 


The most obvious approach to modeling our EMS example of Figure 11.1 is to mini- 
mize the number of locations needed to cover all districts. The following set covering 


model results. 


10 
min > XG 
i=l 


s.t. 


x2 
X1 + X2 
x1 + X3 


X3 


X4 + X5 
x4 + X54 


X4+X5 4 


(LP) 
(IP) 
(NLP) 


(LP) 
(IP) 
(NLP) 


(LP) 
(IP) 
(NLP) 


(number of sites) 


= (district 1) 


>1 (district 2) 
ull (district 3) 
1 (district 4) 
1 (district 5) 
1 (district 6) 
all (district 7) 
= (district 8) 
= il (district 9) 
>1 (district 10) 
>1 (district 11) 
+x > 1 (district 12) 
x7 >1 (district 13) 


(11.8) 
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xg + x9 > 1 (district 14) 
x6 + x9 > il (district 15) 
x5 +%6 > 1 (district 16) 
x5 +x7+ X19 > 1 (district 17) 
xg + x9 > 1 (district 18) 
X9 + X19 > 1 (district 19) 
x19 = 1 (district 20) 
Xi, Se ci 0 onl 


One optimal solution chooses the six sites 2, 3, 4, 6, 8, and 10. That is, 


Xp = X3 = XG HAH =H AQ=X= 1 


Xp SH XE SXF =x =0 


Maximum Coverage EMS Model 


In the Austin case, as in many other real instances, the straightforward covering 
model (11.8) proves inadequate because it calls for too many sites. Suppose that 
we have funds for only 4 EMS locations. How can we find the collection of 4 that 
minimizes coverage insufficiency? 

For this version of the model we need estimates of the demand or importance 
of covering each service district. We will assume the following values have been 


estimated by EMS staff: 
District District District 

i Value i Value i Value 
1 SD 8 DP 15 lS ¥ 5) 

2 4.4 9 7.6 16 25.6 

3} el 10 20.3 ily 11.0 

4 9.0 11 30.4 18 5.3 

5) 6.1 12 30.9 19 7.9 

6 5.7 13 12.0 20 9.9 

i 10.0 14 9.3 


Next we introduce extra decision variables to model uncovered districts i. 


ip Set packing, covering, and partitioning models can be modified to penal-_ 
ize uncovered items i by introducing new variables 
{1 if item is uncovered in the solution 
J = : 
0 otherwis 


into each constraint 7. 


Including such variables, the EMS model becomes 


min 5.2y,; +4.4y2 +7.1y3 + 9.0y4 + 6.1ys (uncovered 
+ 5.7y¥¢ + 10.0y7 + 12.275 + 7.69 + 20.310 district 
importance) 


+ 30.491, + 30.942 + 12.0y)3 + 9. 3y14 ar 15.5y15 
+ 25.6y16 + 11.0917 5.3918 + 7.9yi19 + 9.9 y29 
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st. x2 +y1 Sek (district 1) 

X1 +xX2 + V2 Sill (district 2) 

x1 +x3 +3 Sail (district 3) 

x3 +4 Pall (district 4) 

x3 + ys all (district 5) 

x2 + Ye >1 (district 6) 

Xo +x4+y7 Sal (district 7) 

x3 +X4+ yg >1 (district 8) (11.9) 

xg + yo ull (district 9) 

X4+X6 + V10 >1 (district 10) 

x4 +x5+ yu >1 (district 11) 

Xg+x5t+xX6+t¥n > 1 (district 12) 

Xgt+xst+x7+y3 > 1 (district 13) 

Xg + X9 + 14 Sal (district 14) 

x6 +x9 + Vis > 1 (district 15) 

X5 +xX6 + Y16 >1 (district 16) 

x5 +x7+xX10 +y17 > 1 (district 17) 

Xg +X9 + yis > (district 18) 

X9 + X10 + V19 Syl (district 19) 

X10 + y20 ell (district 20) 

10 

oa a <4 (at most four) 

j=l 

Kiyo es S10e= 0.0 


y1,---,¥20 =O0or 1 
Here the objective function minimizes the total importance of uncovered districts. 
The last (noncovering) main constraint limits solutions to four EMS sites. 
An optimal solution for this more realistic version chooses 
x3 = XG = XE = XU = 1 
MS = va = Vo to 1 
with all other decision variables = 0. That is, sites 3, 4, 5, and 9 are chosen, leaving 


districts 1, 2, 6, 9, and 20 uncovered. The total importance of those districts, which is 
the optimal objective value, equals 32.8. 


ILATING MAXIMUM C 


: SAMPLE EXERCISE 11.8: FORM 


Return to the set covering case of Sample Exercise 11.7(a). Revise the model to 
| maximize the number of algorithms available within a budget of 12. 
| Modeling: We introduce new variables yrp, yrp, and ynrp that = 1 if the indicated 
' algorithm is not provided, and = 0 otherwise. Then the required model is 
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min ypp + yrp + YNLP 


s.t. Xy+%2+%34+X%4+ yp = 1 (LP) 
X2 + X4 + Yip = 1 (IP) 
x3 +%4 + YNLP nl (NLP) 
3x, + 4x2 + 6x3 +14x4 < 12 (budget) 


Xie = Oors 
Yip. ir, YNLp = O or 1 


Column Generation Models 


Another common family of set packing, covering, and partitioning models are de- 
rived from problems involving a combinatorially large array of possibilities too com- 
plex to be modeled concisely. Column generation adopt a two-part strategy for such 
problems. 


1113 | Column generation approaches deal with complex combinatorial prob- 
lems by first enumerating a sequence of columns representing viable solutions 
to parts of the problem, and then solving a set partitioning (or covering or 
packing) model to select an optimal collection of these alternatives fulfilling all 
problem requirements. 


The convenience of this approach comes from its flexibility. Any appropriate 
scheme—however ad hoc—can be employed to generate a rich family of columns 
meeting complex and difficult-to-model constraints. Optimization is reserved for 
the second part of the strategy, when a well-formed model over the columns can be 
addressed by standard ILP technology. 


EXAMPLE 11.4: AA CREW SCHEDULING 


A classic application of column generation arises in the enormously complex problem 
of scheduling crews for airlines. For example, American Airlines* reports spending 
over $1.3 billion per year on salaries, benefits, and travel expenses of air crews. Care- 
ful scheduling, or crew pairing as it is called, can produce enormous savings. 

Figure 11.2 illustrates a (tiny) sequence of flights to be crewed in our fictitious 
case. For example, flight 101 originates in Miami and arrives in Chicago some hours 
later. 

Each pairing is a sequence of flights to be covered by a single crew over a 2- to 
3-day period. It must begin and end in the base city where the crew resides. 

Table 11.3 enumerates possible pairings of flights in Figure 11.2. For example. 
pairing j = 1 begins at Miami with flight 101. After a layover in Chicago, the crew 
then covers flight 203 to Dallas—Ft. Worth and then flight 406 to Charlotte. Finally. 
flight 308 returns them to Miami. 

In real applications, intricate government and union rules regulate exactly 
which sequences of flights constitute a reasonable pairing. Complex software is 
employed to generate a list such as that of Table 11.3. Here we simply allow all 
closed sequences of 3 or 4 flights in Figure 11.2. 


3Based on R. Anbil, E. Gelman, B. Patty, and R. Tanga (1991), “Recent Advances in Crew-Pairine 
Optimization at American Airlines,” Interfaces, 21:1, 62-74. 
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MIA 
(101) (109) 
CHI 
(212) 
(402) (204) (310) 
CHR 
(203) (211) 
DFW 


FIGURE 11.2 Flight Schedule for AA Example 


TABLE 11.3 Possible Pairings for AA Example 


Jj. Flight Sequence — Cost Flight Sequence 

1  101-203-406-308 2900 305-407-109-212 

2 101-203-407 2700 | 10 308-109-212 2050 
3 101-204-305-407 2600 | 11 402-204-305 2400 
4 101-204-308 3000 | 12 402-204-310-211 3600 
5 203-406-310 2600 | 13 406-308-109-211 2550 
6 203-407-109 3150 | 14 406-310-211 2650 
7  204-305-407-109 2550 | 15 407-109-211 2350 
8 204-308-109 2500 


Pairing costs are equally complex. On duty crews are guaranteed pay for minimum 
periods, regardless of the part of that time they are actually flying. If the pairing 
requires overnight stays away from home, hotel and other expenses are added. Values 
for our illustration are shown in Table 11.3. 


Column Generation Model for AA Example 


Having enumerated a list of alternatives like Table 11.3, the remaining task is to find 
a minimum total cost collection of columns staffing each flight exactly once. Define 
decision variables 


a 
ae || (0) otherwise 


| dl if pairing j is chosen 
Then the following set partitioning model does the job: 
min 2900x; + 2700x2 + 2600x3 + 3000x4 + 2600x;5 
+ 3150x656 + 2550x7 + 2500xg + 2600x9 + 2050x109 
+ 2400x141 + 3600x132 + 2550x413 + 2650x174 + 2350x415 
st. Xy $xX2 +%3+%X4 fs —al (flight 101) 
X6 +X7 $Xg + xX9 + X19 $413 4X15 = 1 (flight 109) 
Xy $X2 + X5 + XG il (flight 203) 


(11.10) 
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X3+X4$X7 + XG +44, +X = 1 (flight 204) 


X12 +413 + X14 + X15 =1 (flight 211) 
X9 + X10 —ill (flight 212) 
x34+%7 +x) + x11 =i (flight 305) 
xy + x4 +x%g + X19 $213 = hf (flight 308) 
X54 X12 + X14 = (flight 310) 
xy +X = ll (flight 402) 
Xp tx5 +213 4+ X14 all (flight 406) 
Xo +x3+x% +47 +xX9+xX15 =1 (flight 407) 
My. -2.5.X15°= Olor 1 


An optimal solution makes 
Spina ttn TRE ORE vy oh 
x] =Xg=Xj2=1 


and all other = 0 at total cost $9100. 


SAMPLE EXERCISE 11.9: FORMING COLUMN GENERATION MopELS 


A moving and storage company is allocating 5 long- distance moving loads to cruel 
One feasible combination covers loads 1 and 3 in a 4525-mile route; a second com- 
bines loads 2, 3, and 4 in 2960 miles; a third hauls loads 2, 4, and 5 in 3170 miles; and 
a fourth covers load 1, 4, and 5 in 5230 miles. Form a set partitioning model to decide 
a minimum distance combination of routes covering each load exactly once. 


Modeling: We use the decision variables 


xj = 


nelfnel if route j is chosen 
0 otherwise 


Then the required model is 


min 4525x; + 2960x2 + 3170x3 + 5230x4 (total miles) 


St. xy +xX4 = (load 1) 
x2 + x3 = il (load 2) 
Xx +X2 = (load 3) 
x2 +%34+x4 =1 (load 4) 
x3 4+%4 = il (load 5) 
SGinsa cee = Oyo i 


17,4 ASSIGNMENT AND MATCHING MODELS 


We have already encountered assignment problems in network flow Section 10.5. 
The issue is optimal matching or pairing of objects of two distinct types—jobs te 
machines, sales personnel to customers, and so on. In this section we elaborate on 2 
number of variations that cannot be solved by network flow methods. 


11.4 Assignment and Matching Models 


Assignment Constraints 
It is standard to model all assignment forms with the decision variables 


1 if i of the first set is matched with j of the second 
0 otherwise 


A 
Xij= 


Then corresponding assignment constraints merely require that each object of each 
set be paired exactly once. 


The first system of constraints forces every i to be assigned. The second does the 
same for every j. 


CAM Linear Assignment Example Revisited 


Section 10.5’s computer-aided manufacturing (CAM) model illustrates one assign- 
ment case. Table 11.4 repeats total transportation, queueing, and processing times 
for 8 pending jobs on 10 workstations to which they might next be routed. Each 
workstation can accommodate only one job at a time. We want to find a minimum 
total time routing. 


TABLE 11.4 Transportation and Processing Times for 


CAM Example 
Next Workstation, j 

Job,i 1 2 3 4 5 6 7 8 9 10 
1 8 == 28 
2: — Us es |e IE) 
3 — — 20 — 13 6 — So 
4 —- — — — 19 10 — ~—~ ~—~ — 
5 —- —- — 8 — — 12 — — 16 
6 14 8 3 
dL 6 Diy: 12 
8 — Ss 32 


After introducing dummy jobs 9 and 10, so that we have the same number of 
jobs as machines, this problem clearly has the assignment form. A complete model 
is given in (10.6) of Section 10.5. Our interest here is in the objective function 


min 8x14 + 23x13 + 5x19 + 4x22 + 12x24 + 15x2,5 
+ 20x33 + 13x35 + 6x36 + 8x3,.g + 19x45 + 10X46 
+ 8x54 + 12x57 + 160x510 + 14X61 + 8X6,7 + 3X6,9 


| (11.11) 
+ 6X7,2 + +27x7,3 + 12x7,10 + 5xg.2 + 15xg3 + 32x88 


Sil) 
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Linear Assignment Models 


CAM model (10.6) is a linear assignment model because its objective function (11.11) 
is linear. 


SAMPLE EXERCISE 11.10: FORMULATING LINEAR ASSIGNMENT MODELS | 


A swimming coach is choosing his team for a medley relay. One swimmer will swim 
the back stroke leg j = 1, one the breast stroke leg j = 2, one the butterfly leg j = 3. 
and one the free-style leg j = 4. From previous experience the coach can estimate 
the time, ¢;;, that swimmer i could achieve on leg j. Formulate a linear assignment 
model to choose the fastest medley team. 


Modeling: Using the decision variables 


od if swimmer i swims leg j 
Xij = , 
: 0 otherwise 


the required model is 


4 4 
min SE Ss HX; (team time) 
s.t. Dis Hy Al Deel, ee (one leg per swimmer) 


ey Lip fe len (one swimmer per leg) 


x7 = O'or 1 i Se aR PHA, cso?! 


Swimmers are assigned to relay legs to minimize total team time. 


Quadratic Assignment Models 


What produces the linearity in definition is that the objective function weights 
single decisions to pair 7 in one set with j in the other. For example, in objective 
(11.11), a decision to assign job 4 to workstation 5 adds 19 to the total time of the 
solution, regardless of how other decisions are resolved. 

Many assignment problem circumstances do not fit the linear case because 
the objective function depends on combinations of decisions. That is, the impax 
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of one decision cannot be assessed until we know how others are resolved. Such 
circumstances often lead to quadratic assignment models. 


11.16 | Quadratic assignment models minimize or maximize a uadratic objec : 
tive fonehon of the pie : 


of assigning i to ME and k to £. 


Notice that each objective function term 
Cijke *Xij * Xk,e 


involves two assignment decisions. Cost cjj%,¢ is realized only if both x;; = 1 and 
xx = 1. That is, c;;x,¢ applies only if i is assigned to j and k is assigned to £. 


EXAMPLE 11.5: MALL LAYOUT QUADRATIC ASSIGNMENT 


Some of the most common cases producing quadratic assignment models arise in 
facility layout. We are given a collection of machines, offices, departments, stores, 
and so on, to arrange within a facility, and a set of locations within which they must 
ft. The problem is to decide which unit to assign to each location. 

Figure 11.3 illustrates with 4 possible locations for stores in a shopping mall. 
Walking distances (in feet) between the shop locations are displayed in the adjacent 
table. The 4 prospective tenants for the shop locations are listed in Table 11.5. The 
table also shows the number of customers each week (in thousands) who might wish 
to visit various pairs of shops. For example, a projected 5 thousand customers per 
week will visit both 1 (Clothes Are) and 2 (Computers Aye). 


Distance (feet) 
1 2 3 4 


J 
1 _ 80) = 150)- 170 
2 80 — 130 100 
3 4), 150» 130 — 120 
4.) 1508) 1005). 120) — 


FIGURE 11.3. Mall Layout Example Locations 


Mall managers want to arrange the stores in the 4 locations to minimize cus- 
‘omer inconvenience. One very common measure is flow-distance, the product of 
Sow volumes between facilities and the distances between their assigned locations. 
For example, if shop 1 (Clothes Are) is located in space 1, and shop 4 (Book Bazaar) 
s located in space 2, their 7 thousand common customers will have to walk the 80 
‘cet between the locations. This adds 7 - 80 = 560 thousand customer-feet to the 
Sow-distance. 


578 Chapter Eleven e Discrete Optimization Models 


TABLE 11.5 Mall Layout Example 


Tenants 

Common Customers 
with k (000’s) 

Store, i 1 2 3 4 
1: Clothes Are — 5 2 7 
2: Computers Aye si5 = 3) 8 
3: Toy Parade 2) 3 = 3 
4: Book Bazaar 7 8 30 — 


Mall Layout Example Model 


Notice that the flow-distance for any pair of shops cannot be computed until we 
know where both are assigned. This is the assignment combinations characteristic 
that yields quadratic assignment models. 


Using the decision variables 
aire if shop is assigned to location j 
Xij= : 

0 otherwise 


the required quadratic assignment model is 
min 5(80x1,1%2,2 + 150x1,1%2,3 + 170x11x2,4 (shops 1 and 2) 
+ 80x1,2%2,1 + 130x1 2x23 + 100X224 
+ 150%1,3%2,1 + 130x1,3%2,2 + 120%1,3x2,4 
+ 170%} ,4X2,1 + 100%1,4%2,2 + 120% 4x23) 
2(80x1,1%3,2 + 1502x1,1%3,3 + 170%1,1%3,4 (shops 1 and 3) 
+ 80x1,2%3,1 + 130x1,2%3,.3 + 100X1,2%3,4 
+ 150x1,3%3,1 + 130x1,3%3,2 + 120x1,3x3,4 
+ 170x1,4x3,1 + 100%1,4%3,2 + 120x1 4x33) 
7(80%4,1%4,2 + 150%1,1.%4,3 + 170%1,1%4,4 (shops 1 and 4) 
+ 80x1,2.%4,1 + 130X1,2%4,3 + 100%1,2%4,4 
+ 150x1,3%4,1 + 130x1,3%4,2 + 120x1,3%4.4 
+ 170% ,4X4,1 + 100%1,4%4,2 + 120x1,4%43) 
3(80x2,1X3,2 + 150x2,1x3,3 + 170%2,1%3,4 (shops 2 and 3) 
+ 80x2,2%3,1 + 130x2,2%3,3 + 100x2,2.%3.4 
+ 150x2,3x3,1 + 130x2,3x3,2 + 120x23x3.4 
+ 170x2,4x3,1 + 100x2,4%3,2 + 120x2,4x3,3) 
80x2,1%4,2 + 150%2,1%4,3 + 170x21%4,4 (shops 2 and 4) 
+ 80x2,2%41 + 130x2,2x43 + 100X2,2%4,4 
+ 150x2 3X41 + 130x2 3X42 + 120x2 3X44 
+ 170x2,4%4,1 + 100X2,4%4.2 + 120x2,4%4,3) 


(11.12 


8 


= 
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3 (803,142 + 150x3,1%4,3 + 170%3,1%4.4 (shops 3 and 4) 
+ 80x3,2%4,1 + 130x3,2x%4,3 + 100x3,2%4.4 

+ 150x3 3x41 + 130x3.3%4.2 + 120x3,3%4,4 

+ 170x3,4X4,1 + 100X3,4%4,2 + 120x3,4%4,3) 


St. X11 $212 + 413+ 414=1 (1, Clothes Are) 
Xoq X99 eo 3: ep eal (2, Computers Aye) 
X31 + 2X32 +%33+%34=1 (3, Toy Parade) 
X41 + X42 +%43+X44=1 (4, Book Bazaar) 
UG dese og Sees te a — 1 (location 1) 
OY ee) OSI ee oy — it (location 2) 
X13 +23 +%33+%43 = 1 (location 3) 
X14 + X24 +%3,4+X44=1 (location 4) 


xj =Oorli=1,...,4j=1,...,4 


The objective function computes total flow distance for all pairs of shops and all 
possible assigned locations. Assignment constraints assure that one shop goes to 
each location and each locations gets one shop. An optimal assignment places shop 
| in location 1, shop 2 in location 4, shop 3 in location 3, and shop 4 in location 2, for 
a total flow distance of 3260 thousand customer-feet. 


squares, g, each of which will be the location of a single machine m. He has also es- 
timated the distance, d, »/, between all pairs of grid squares and the number of units, 
fn.’ that will have to travel between machines m and m’ (in both directions) during 
each week of operation. Formulate a quadratic assignment model to layout the shop 
in a way that will minimize material handling cost (i.e., minimize the product of be- 
tween machine flows and the distance between their locations). Assume dy.” = dy,g. 


Modeling: Using the decision variables 


1 if machine m is located at grid square g 
Xm,g = a 
0 otherwise 


the required model is 


SS AI) 
min Se a Se a Sim Ug, ¢'Xm,gXm',g' (flow distance) 


m=1 g=1 m'>m g/=1 
a'#8 


12 
st. SS ees id Sees 12 (square per machine) 
g=l 
12 
eS Sing all Pi — alee ele (machine per square) 
m=1 


xij =Oorl PEN ae De a OL acca MD 
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Machines are assigned to grid squares to minimize the total flow distance which 
measures the material handling implication of a layout. Index ranges in the objec- 
tive function assure that each pair of machines and locations is reflected just once. 


Generalized Assignment Models 


Main assignment constraints of definition | 11.14] require, respectively, that each 
object i of one set is assigned to exactly one j of the other, and that each j receives 
one i. Suppose, instead, that each object i must be assigned to some j, but that j’s 
may receive several i. Specifically, define 
b; A capacity of j 

; 4 size, space, or similar amount of j’s capacity consumed if 7 is assigned to j 

4 cost (or benefit) of assigning i to j 
Then finding the best way to assign all i without violating capacities is called a gen- 
eralized assignment model. 


11.17 | Generalized assignment models, which encompass cases where allocation 
of i to j requires fixed size or space s;; within j capacity b;, have the form 


min or max > Cj Lig 
Z 


s.t. y xj =1 for all i 
: oe 


> Sip Xiy < d; for all j 
i 


4; = Qorl for all 7,7 


Here c;; is the cost (or benefit) of assigning i to j and all sums are limited to 
(i, }) combinations allowed in the problem. 


EXAMPLE 11.6: CDOT GENERALIZED ASSIGNMENT 


The Canadian Department of Transportation encountered a problem of the gen 
eralized assignment form when reviewing their allocation of coast guard ships on 
Canada’s Pacific coast.4 The ships maintain such navigational aids as lighthouses 
and buoys. Each of the districts along the coast is assigned to one of a smaller num- 
ber of coast guard ships. Since the ships have different home bases and differen 
equipment and operating costs, the time and cost for assigning any district varies 
considerably among the ships. The task is to find a minimum cost assignment. 

Table 11.6 shows data for our (fictitious) version of the problem. Three ships— 
the Estevan, the Mackenzie, and the Skidegate—are available to serve 6 districts 
Entries in the table show the number of weeks each ship would require to main 
tain aides in each district, together with the annual cost (in thousands of Canadias 
dollars). Each ship is available 50 weeks per year. 


“Based on Joseph G. Debanne and Jean-Noel Lavier (1979), “Management Science in the Pus» 
Sector—the Estevan Case,” Interfaces, 9:2, part 2, 66-77. 


ee eee ee ee ee ee 
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TABLE 11.6 Costs and Times for the CDOT Example 


District, i 
Ship, j 1 2 3 4 5 6 


1: Estevan Cost 130 30) a0 30 340 20 

Time 30 50 10 11 13 9 

| 2: Mackenzie Cost 460 150 20 40 30 450 
Time 10 20 60 10 10 17 

3: Skidegate Cost 40 370 120 390 40 30 

Time 70 10 10 15 8 12 


CDOT Example Model 


Using the decision variables 


real a. if district i is assigned to ship j 
Pag ; 
eae baa otherwise 


this CDOT example can be formulated 

min 130x1,1 + 460x1,2 + 40x1,3 + 30x2,1 + 150x9,9 + 370x2,3 
+ 5103, + 20x32 + 120x353 + 30x41 + 40x42 + 390x43 
+ 340x5,1 + 30x5,2 + 40x5,2 + 20261 + 450%6,2 + 30X63 


st. X11+4%12+2%13=1 (district 1) 
X21 + X22 +%23=1 (district 2) 
X31 + 3,2 +233 =1 (district 3) 
X41 +%4,2 +X43 =1 (district 4) 
X51 +xX52+xX53=1 Kdistaict 2) (11.13) 
Kota XG et .X6. 4 (district 6) 
30x11 + 50x21 + 10x31 (Estevan) 
+ 11x41 + 13x51 + 9x61 <50 

10x1,2 + 20x22 + 60x3,2 (Mackenzie) 
+ 10x42 + 10x52 + 17x62 < 50 

70x1,3 + 10x23 + 10x33 (Skidegate) 


+ 15x43 + 8x53 +12x%63 <50 
xij =Oorl tee fale eS 


The objective function minimizes total cost. The first 6 constraints assure that every 
district is assigned to one ship, and the last 3 keep work assigned to each ship within 
the 50 weeks available. An optimal solution assigns districts 1, 4, and 6 to the Estevan, 
districts 2 and 5 to the Mackenzie, and district 3 to the Skidegate at a total cost of 
$480,000. 


Objects i = ie , LOO of “sin ne Gi cubic meters are pees iene in an automated 
warehouse. Sion locations j = 1,..., 20 are located d; meters from the system’s 
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input/output station, and all have capacity b cubic meters. Formulate a generalized 
assignment model to store all items at minimum total travel distance assuming that 
as many objects can be placed in any location as volume permits. 


Modeling: Using the decision variables 


is | 1 if object i is stored in location j 
xij = 


0 otherwise 


the required generalized assignment model is 


min 1D ye djxij (total distance) 


Sy Ki i=1,..., 100 (each i stored) 


» Gequda oy) — Nes A0 (j capacity) 


xij =Oorl ally eee LOO ij She gon 20 
The objective function totals move distance to assigned locations, the first system 


of main constraints assures that every object is stored, and the second enforces 
capacities. 


Matching Models 


Assignment problems considered so far always pair objects in two distinct sets. One 
final variation eliminates the distinction between the sets. Decision variables of such 
matching models are 

1 if iis paired with 7’ 


Xi,i S 


0 otherwise 


where by convention index i’ >i to avoid double counting. 


11.18 | Matching models, 
the form 


The two sums in each main constraint of formulation | 11.18] are required becaus: 
any particular i will be the higher index in some pairs and the lower index in others 


11.4 Assignment and Matching Models 


Matching models include linear assignment cases [1145 if allowed pairings 
are restricted to those matching an object in one class with an object in another. 
However, the matching term is usually reserved for the more general case where all 
objects come from a single class. 


EXAMPLE 11.7: SUPERFI SPEAKER MATCHING 


We may illustrate matching with the task faced by fictitious high-fidelity speaker 
manufacturer Superfi. Superfi sells its speakers in pairs. Even though the manu- 
facturing process maintains the most rigid quality standards, any two speakers pro- 
duced will still interfere slightly with each other when connected to the same stereo 
system. 

To improve its product quality even more, Superfi has measured the distortion 
di for each pair of speakers in the current lot. They wish to determine how to pair 
the speakers so that total distortion is minimized. 

Notice that any two speakers may be paired. There is no distinction between 
large and small, or left and right. 


Superfi Example Model 


We may model this problem with the decision variables 
1 if speakers i and 7’ are paired 


Xi = 2 
0 otherwise 


There is one for each pair (i, i’), i <i’. The corresponding matching model | 11.18 | is 


min Se ‘B dj iXji7 (distortion) 
i i>i 
st. Da xii + a xiv =1 for alli (each speaker paired) (11.14) 
r<i ii 
x7 =O0orl for alli, i’ >i 


The objective function sums the distortion of all selected pairs, and main constraints 
assure that each speaker is part of exactly one pair. 


RMULATING MATCHING MODELS 


The instructor in an operations research class is assigning his students to 2-person 
teams for a term project. Each student s has scored his or her preference p,.y for 
working with each other student s’. Formulate a matching model to form teams in a 
way that maximizes total preference. 


Modeling: Using the decision variables 


Re 1 if 7 is teamed with i’ 
fon 
Bre) he) otherwise 


the required matching model is 
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max Ss ys (Pie + Di) Xii (preference) 
i >i 
s.t. >» Xi + ye MG for alli (each student paired) 
v<i v>i 
Xiv =Oorl foralli,i’ >i 


Each term of the objective captures the two-way preference gain of teaming 7 with 
i’, and the main constraints assure that each student is assigned to one team. 


Tractability of Assignment and Matching Models 


The various models of the assignment family presented in this section provide a 
good illustration of the tremendous variation in tractability of discrete optimization 
models. 


i. 19 | Linear assignment models a ; eke tractable because [oe can ibe viewed 


"ate size ae ihe methods of Sections 12. 2 to 12.5. sui hey are far less tractable 
than the linear assignment case. — 


instances. 


1 1.5 TRAVELING SALESMAN AND ROUTING MODELS 


Among the most common discrete optimization problems are those that organize 
a collection of customer locations, jobs, cities, points, and so on, into sequences or 
routes. Sometimes such routing models form all points into a single sequence. Other 
times, several routes are required. 


Traveling Salesman Problem 


The simplest and most famous of routing problems is known to researchers as the 
traveling salesman problem. 


11.5 Traveling Salesman and Routing Models 


The name derives from a mythical salesperson who must make a tour of the 
cities in his or her territory while traveling the least possible distance. TSPs actually 
occur in a much wider expanse of applications. Any task of sequencing objects in 
minimum total cost, length, or time can be viewed as a traveling salesman problem. 


EXAMPLE 11.8: NCB CIRCUIT BOARD TSP 
A practical scenario for thinking about traveling salesman problems occurs in the 


manufacture of printed circuit boards for electronics.° Circuit boards have many 
small holes through which chips and other components are wired. Ina typical exam- 
ple, several hundred holes may have to be drilled in up to 10 different sizes. Efficient 
manufacture requires that these holes be completed as rapidly as possible by a mov- 
ing drill. Thus for any single size, the question of finding the most efficient drilling 
sequence is a traveling salesman routing problem 
Figure 11.4 shows the tiny example that we will investigate for fictional board 
manufacturer NCB. We seek an optimal route through the 10 hole locations indicated. 
Table 11.7 reports straight-line distances d;; between hole locations i and j. Lines in 
Figure 11.4 show a fair quality solution with total length 92.8 inches. The best route 


is 11 inches shorter (see Section 12.6). 
e 6 
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FicurRE 11.4 Board Drilling Locations for NCB Example 


Symmetric versus Asymmetric Cases of the TSP 


585 


An important distinction among traveling salesman problems concerns whether dis- 


ances between points are symmetric or asymmetric 


*Based in part on Surya Danusaputro, Chung-Yee Lee, and Louis A. Martin-Vega (1990), “An 
=iicient Algorithm for Drilling Printed Circuit Boards,’ Computers and Industrial Engineering, 18, 


5-151. 
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TABLE 11.7 Distances between Holes in NCB Example 


5 


1 2 3 4 5 6 7 8 9 10 
_— 3.6 Slee 10:0 153) G20. 0cnMR0 TI42s! 23:01 2:26:4 


oe 


1 

2 3.6 — 3.6 64 12d 8B? 1016 197 = 23:0 
3 5.1 3.6 — TA ¢ alOL6. 15.0!  SUS:8S TOS 184 © 92159 
4 10.0 6.4 al _— 7.0) 15us, SLO AD ABO 170 
5) 153) 12iin e016 7.0 — 919" ail5:3 5.0 Teoh coals) 
6 20:0) 518.15, 2415/0 3 15:7 99 — 250 149 120 15.0 
7 N60. 13:2 415:8)=5 10:0" 5153 25.0 — 103° 1927 21.0 
8 142 106 10.8 4.2 5:04 14.9 5 0S: — 102 13.0 
9 23:0 Fro a eSiAerds:9 SUP le O LOD eel 2 _— 3.6 
0 


R 


264.) 2310 20m 7.0, 1S LS ORO 3:0 3.6 — 


Our NCB example is symmetric because d;,; = dj; in Table 11.7. In other cases 
distances are not symmetric in this way because travel from i toj is with the traffic, ane 
travel from j to i against, or because of a host of similar asymmetric circumstances 


Formulating the Symmetric TSP 


One thing that intrigues researchers about traveling salesman problems is that ther: 
are many different formulations—none of them straightforward. Most ILP modes 
of the symmetric case employ decision variables, i < j, 


1 if the route includes a leg between i and j 
Xij = : 
0 otherwise 
Notice that we define x;; only for i < j. This numbering convention avoids th 
duplication that could result because a leg between i and j implies one between j ans 
i, and costs are the same. 
In terms of these new decision variables, the total route length now has the eas 


linear form 
min e. oS, dighiy 148 


i j>i 


What makes ILP traveling salesman models complex is their constraints. ~ 
little contemplation will make one system clear. In the symmetric case, exactly t 
x-variables relating to any point ican be = 1 ina feasible solution. One links i to ™ 
city before it in the route, and the other links 7 to the city after. We can express 1 
requirement mathematically with constraints 


SL Se for alli (is 


j<i j>i 
A specific instance for i = 5 in the NCB example of Figure 11.4 is 


X15 + X25 +%3,5 + x45 +56 +%5,7 +%5,8 + X59 + x5,10 = 2 


SAMPLE EXERCISE 11.14: MODELING TSPs as ILPs 


The following graph show the available links joining 6 points, with numbers on =) 
indicating transit times. 
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We wish to find the shortest time route visiting all nodes exactly once and using only 
links shown in the graph. 


(a) Explain why this problem can be viewed as a symmetric traveling salesman prob- 
lem. 


(b) Formulate integer linear program objective (11.15) for this instance. 


(c) Formulate constraints (11.16) for this instance. 


Modeling: 


(a) The problem is a TSP because it requires a closed route visiting each point. It is 
symmetric because the time is the same whether a link is passed in the i-to-j or the 
j-to-i direction. 


(b) The linear objective required is 
min 10x12 + 1x13 + 1x1,5 + 1x24 + 1x26 
+ 10x34 + 1x35 + 1x46 + 10x5,6 
which minimizes total tour length. 


c) Needed constraints (11.16) are 


12 +413 4+%1,5 =2 (node 1) 
X12 +x2,4+%26 =2 (node 2) 
X13 $%34 +2435 = g) (node 3) 
X2,4+%34+X46 =2 (node 4) 
MGtusist wears at S16 2 (node 5) 
X2,6+X46+%56 =2 (node 6) 


Subtours 


| Figure 11.5 illustrates why constraints (11.16) are not usually enough. The solution 

shown does have two links at each point. But it divides the 10 hole locations among 

| three subtours or miniroutes. We seek a single route through all the points. 

| No one has any difficulty understanding subtours, but constraints to prevent 
them are less obvious. One form of subtour elimination constraints is obtained from 


any 


S 4 proper subset of the points/cities to be routed 


Every tour must cross between points in S and points outside at least twice. This 
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FIGURE 11.5 Subtour Solution for NCB Example 


leads to constraints of the form 
number of legs 


between pointsinS | = y Ss Xij + y ye yp = 2 


and points not in $ ieS j¢S idS jeS 


There is one such constraint for every proper subset S of at least 3 cities. 
The subtour solution of Figure 11.5 violates several of these constraints. For 


example, pick S 4 {5,6,9,10}. The corresponding subtour elimination constrain’ 
lists all possible tour legs passing into or out of set S: 
X1,5 X16 + %1,9 + X1,10 + X2,5 

+ X2,6 + X2,9 + X2,10 + X3,5 + X3,6 

+ X3,9 + 3,10 + X4,5 + X4,6 + X4,9 

+ X4,10 + X5,7 + X6,7 + X7,9 + 7,10 

+ X5,3 + X6,8 + X8,9 + X8,10 

>2 


(11.17 


SUBTOUR ELIMINATION CONSTRAINTS - 


SAMPLE EXERCISE 11.15: Fo 
Return to the TSP OF Sample Exercise 11.14. 

(a) Show by inspection that the least cost binary solution over = 2 constraints © 
Sample Exercise 11.14(b) has subtours. 

(b) Formulate a subtour elimination constraint (11.17) not satisfied by the subtow 


solution of part (a). 


Analysis: 
(a) It is obvious that the least cost binary solution touching each node with exact 


two links has 


11.5 Traveling Salesman and Routing Models 


X13 = %15 = %35=1 
X24 = %2,6 = X46 =1 
X12 = X34 = %5,6 = 0 
It contains subtours 1-3-5—1 and 24-62. 
(b) One violated subtour elimination constraint (11.17) arises from 
Sy Sail. 3 St 
Summing all edges joining a node in S to one outside gives 


X12 +%3,4+%56 > 2 


ILP Model of the Symmetric TSP 


Combining expressions (11.15)—(11.17) produces a full integer linear programming 
formulation of the symmetric traveling salesman problem: 


=? forall 


- igs JES ji 


for all props: point 


ILP Model of the Asymmetric TSP 


How must formulation | 11.25 | be modified to address the asymmetric case? Several 
insights are required. 


e Inthe asymmetric case we use decision variables 


A 


| il if the tour passes i to j 
Xi = 


0 otherwise 
for all combinations of i and j. With costs asymmetric it matters whether the route goes 
itoj ofj toi. 

e Instead of just meeting each point twice, any asymmetric TSP route must enter each 
point once and leave each point once. Thus = 2 constraints of the symmetric formulation 


become assignment constraints (system 11.14 ) in the asymmetric case: 
SS Yee for alli (enter 7) 


Ds, xij=1 for alli (leave 7) 
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e Each tour must enter and leave every subset S of points. Thus we may accomplis® 
subtour elimination by requiring the tour to leave every S at least once: 


.s SS xij 21 all proper point subsets S 
ieS j¢S 


Combing produces a full asymmetric formulation. 


Return to ae TSP of SE Beeniees 11.14 an 11.15, and assume that ¢ a 2- -une 
cost penalty is required when the tour passes from a higher to a lower node number 
That is, di. = 10, but dz; = 10 +2 = 12, d)3 =1, but d3; = 1+2 = 3, and so om 


(a) Explain why the problem is now an asymmetric TSP. 


(b) Formulate an objective function for the corresponding model | 11.26]. 


(c) Formulate constraints requiring the tour to enter and leave each node exact» 
once. 


(d) Formulate a subtour elimination constraint requiring the tour to leave node subs= 
= {1, 3, 5} at least once. 


Analysis: 
(a) The penalty of 2 makes d;; # dj;, which turns the problem asymmetric. 


(b) Including variables for all one-way passages, the objective function of forme 


min 10x12 + 1413 + 1x1,5 
+ 12x91 + 1x2,4 + 1x26 
+ 3x31 + 10x34 + 1x3,5 
+ 3x42 + 12x43 + 1X4,6 
+ 3x51 + 3x53 + 10x5,6 
+ 3x62 + 3x64 + 12x65 


11.5 Traveling Salesman and Routing Models 


c) These constraints have the assignment form: 


%21+%31+%51 =1 
44,2 +%X42 +%62 = 1 
X134+%43 +453 =1 
X24 +%34+%64 =1 
415 +%35+%65 =1 


X26 +xX46+%56 =1 


plus 


12 +4%13+%15 =1 
X21 +X24+%26 =1 
X31 +%34+2%35 =1 
X42 +%43 +X46 =1 
X51 +%53+%56 = 1 
x62 +. %6,4 +X65 =1 


d) To avoid any subtour among nodes in S$ = {1, 3, 5}, we add subtour elimination 
constraint. 


X12 +%34+4%56>1 


Quadratic Assignment Formulation of the TSP 


There are an enormous number of subtour elmination constraints for a TSP on evena 
modest number of points. That is why it is sometimes easier—especially in developing 
teuristic procedures of Section 12.6 and 12.7—to deal with a TSP formulation having 
simpler constraints. We can accomplish this, at the cost of making the objective 
‘unction nonlinear, by formulating the TSP as a quadratic assignment (QAP) model 
11i6). 

Think of the tour as a sequence or permutation of the points to be visited. 

Decision variables in the QAP form assign sequence positions k to points i; that 


>, 


me if kth point visited is i 
Vki = 


0 otherwise 


For example, the illustrative route of Figure 11.4 has 


Jil = 2,3 = ¥3,6 = 45 = 5,4 = Y6,8 
= Y7,9 = Y8,10 = 9,7 = Y10,2 
=Al 


when viewed as starting at hole 1. In terms of these variables, either the symmetric 
or the asymmetric case can be formulated as an INLP. 
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Dn. Ye = 7 for alli | point visited) 


= Sho for ake 


where y,; = 1 if the kth visited point is i, and dj; 2 distance from point i to 
point /. 


Constraints in| 11,27 | have the usual assignment format | 11.14), but the objec- 


tive function is less transparent. Terms 
dij X VRiVkH j 


add in distance dj; exactly when j follows i somewhere in the chosen tour sequence. 
(Here we take k + 1 to mean | when k is the highest-numbered point.) In Figure 
11.4, for example, one of the nonzero terms would be 


d3,6 + Y2,3 + V3.6 


because holes 3 and 6 could be second and ee in sequence. Summing over all i and 
j recovers total distance. 


Problems Requiring Multiple Routes 


For many trucking and other distribution organizations, the problem is to design 
many routes, not just one. The task begins with a collection of stops to be serviced 
Decisions first subdivide the stops into several routes and then choose the visitation 
sequence for each route. 


EXAMPLE 11.9: KI TRUCK ROUTING 


Kraft Incorporated confronts such multiple-route design problems in planning truck 
delivery of its food products to over 100,000 commercial, industrial, and military 
customers in North America.° Known customer requirements must be grouped inte 
truckloads and then routes planned. 

Our tiny fictitious version of the KI case is illustrated in Figure 11.6. The 20 stops 
indicated are to be serviced from a single depot. Table 11.8 displays the requirements 


6Based on Hong K. Chung and John B. Norback (1991), “A Clustering and Insertion Heuristic 
Applied to a Large Routing Problem in Food Distribution,” Journal of the Operational Research Societ 
42, 555-564. 
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at each stop, expressed in fractions, /;, of a truckload. The 7 routes depicted in Figure 
11.6 show one feasible solution. 


1@---@3 


20 


FIGURE 11.6 Depot and Delivery Locations 
in the KI Example 


TABLE 11.8 Fractions of Truckloads to Be Delivered in KI Example 


Stop,i Fraction, f; | Stop,i Fraction, f; | Stop,i Fraction, f; i Fraction, f; 


KI Truck Routing Example Model 


With complex problems like KI’s it is not easy to see where to begin a model. 


in n optimization models 


We will start with the assignment of stops to routes. The decision variables 


is | il if stop 7 is assigned to route j 
Zij= 


0) otherwise 
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Generalized assignment constraints (definition| 11.17} ) then manage the allo- 
cation of the 20 stops to seven routes: 


al 
a Lie formally =1, 22,20 (each i to some /) 


20 (11.18 
aps formally: = 1, xe 7 (truck capacities) 


Zij =Oorl forallya= 1, yee. 20 aa eee 


The first set makes sure every stop goes to some route, and the second keeps loads 
to assigned trucks within capacity. 

The difficulty of concisely formulating routing problems becomes apparent 
when we try to express an objective function to go with (11.18). Certainly, it is 
something like 


7 
> 6(2) (11.19 
j=l 


where z is the vector of z,;; and 


6;(z) & length of the best route through stops assigned 
to truck j by decision vector z 


However, functions like 6;(z) are highly nonlinear. In fact, 6;(z) is the optimal solution 
value of a traveling salesman problem over the stops allocated to route j. 

No concise expression of such functions exists. Still, we know what they mean 
in the problem context. Section 12.8 will show that good heuristic solutions can be 
obtained with rough approximations. 


11.6 FACILITY LOCATION AND NETWORK DESIGN MODELS 
Principles and} 


modeled iiE new binary variables and switching constraints. Virtually any linear 
program with fixed charges can be modeled in that way, but some forms are especially 
common. In this section we introduce the classic facility location and network design 
cases. 


.3| of Section 11.1 demonstrated how fixed charges can be 


Facility Location Models 


Facility location models, also called warehouse location models and plant location 
models, are perhaps the most frequently solved of all forms involving fixed charges. 


11.29 | 
of facilities to open in order t 
total cost. 2 


Costs include both the variable cost of servicing customers from chosen facilities anc 
the fixed cost of opening the facilities. 
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EXAMPLE 11.10: TMARK FACILITIES LOCATION 


AT&T has confronted many facility location problems in recommending sites for 

the toll-free call-in centers of its telemarketing customers.’ Such centers handle tele- 

phone reservations and orders arising in many geographic zones. Since telephone 

rates vary dramatically depending on the zone of call origin and the location of the re- 

_ ceiving center, site selection is extremely important. A well-designed system should 
minimize the total of call charges and center setup costs. 

Our version of this scenario will involve fictional firm Tmark. Figure 11.7 shows 
the 8 sites under consideration for Tmark’s catalog order centers embedded in a map 
of the 14 calling zones. Table 11.9 shows corresponding unit calling charges, r;;, from 
each zone j to various centers i, and the zone’s anticipated call load, dj. 


i Fixed Cost 
2400 
7000 
3600 
1600 
3000 
4600 
9000 
2000 


DrADNHWNK |] 


FIGURE 11.7 Customer Zones and Possible Facility Locations for Tmark 
Example 


TABLE 11.9 Unit Call Charges and Demands for Tmark Example 


Possible Center Location, i 


Mia Se “eee 2 Dee ee Call 
Zone, j 1 2 3 4 5 6 7) 8 Demand 

1 1e2smeerie4 OOM (090 SaS0 1290" 2:00! 240 250 
2, OLSOM 0:90 O90 130404 2:20) 207 31080 150 
3 0.70 0.40 0.80 1.70 1.60 2.50 2.05 1.60 1000 
4 O90 V1-20) 1.40) 70:50, 155" -1:70) =1!80' 1.40 80 
5 0:80.-..0:70,, 0:60), O%70.,. 1.45). 1.80... 1.70: 1.30 50 
6 TORS 170% TO 0605 20.908 “1.30 arie30 171.40 800 
7 1.40 140 125 080 0.80 1.00 1.00 1.10 325 
8 1-305 1502 OOM A LOSINOI.O Fee50) VSORE 1.00 100 
9 1.50 1.90 1.70 1.30 040 0.80 0.70 0.80 475 
10 1S ee 160) 1-30 150 SOO 10) a0) 10.70 220 
11 2.10 2.90 240 1.90 1.10 2.00 0.80 1.20 900 
12 1.80 2.60 2.20 1.80 0.95 0.50 2.00 1.00 1500 
13 1.60 2.00 1.90 1.90 140 1.00 0.90 0.80 430 
14 200m 240R. 2.00)| 4 2201 S150) A220). 110) 0:80 200 


7Based on Thomas Spencer III, Anthony J. Brigandi, Dennis R. Dargon, and Michael J. Sheehan 
1990), “AT&T’s Telemarketing Site Selection System Offers Customer Support,” Interfaces, 20:1, 83-96. 
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Any Tmark center selected can handle between 1500 and 5000 call units per day. 
However, their fixed costs of operation vary significantly because of differences in 
labor and real estate prices. Estimated daily fixed costs, f;, for the 8 centers are 
displayed in Figure 11.7. 


ILP Model of Facilities Location 


Clearly, there are two kinds of decisions to make in facilities location: which facilities 
to open, and how chosen facilities should serve customer demands. We employ the 
decision variables 
rat if facility i is opened 
Mae 0 otherwise 
to determine which sites are selected. A second set, 
xij & fraction of customer j demand satisfied from facility i 


decides how service is to be allocated. Combining produces a standard model of 
facilities location: 


11.30 Basic facilites location problems can be formulated as the integer linear 


program 
min = > Cr jaa “- » Yi (total cost) 
oe i 
go ee for all j (fulfill } demand) 
i - 
a AjXij = Uy: ~ for alli (i ‘capacity switching) : 


for alli 


nonnegative fixed cost of opening facility 7, and u; is the capacity of facility i. 


This objective function sums all variable and fixed costs. The first system oF 
constraints assures that 100% of each customer demand is fulfilled. The second se 
switches “on” facility capacity when the corresponding y; = 1. If the problem has 
no true capacities on facilities, any sufficiently large value can provide a uj—perhaps 
total demand. 

Sometimes customers must be 100% serviced from a single facility, so thay 
constraints 


jae OTll for all 7,7 


are also enforced. Other times, demand can be split among several facilities, which 
leaves x;; continuous. 
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Tmark Facilities Location Example Model 


Following format) 11.30], we can express a Tmark’s facilities location problem in the 
mixed-integer linear program 


8 14 8 

min YO (jaa + >) fi (total cost) 
i=1 j=l i=l 
8 

st. iy = forallj=1,...,14 — (carry j load) 
i=l 


14 
1500y;< S\djx;;  foralli=1,...,8 minimum ati) (11.20) 
tied Dy) 
j=l 


14 
yy ajx;j<5000y;  foralli=1,...,8 (maximum at 7) 


j=l 
xij = 0 forall. gle ee8s 2) — Mh, 14 
y; =Oorl foralli=1,...,8 


Here the objective function sums calling and setup costs. The total cost of call 
load serviced by i for zone | is 


(demand at j)(i, j telephone rate)(fraction of j serviced from i) = djrjjxi; 


Most constraints of model (11.20) are identical to pattern | 11.30]. Variables 
*,; are continuous because it is assumed that the load from a zone can be split among 
several centers. 

The one new element is switching constraints dealing with minimum operating 
levels. Constraints 


14 
1500y; <—s 3 djXij 
j=l 


enforce the requirement that any open center 7 must handle at least 1500 call units 
per day. 
Solution of model (11.20) produces 


Yi = V2 = V3 = V5 = VE = V7 =O 


with a total cost of $10,153 per day. Zones 1, 2, 4, 5, 6, and 7 are serviced from center 
+. and the rest from center 8. 


country from which inspectors would make annual visits to each of 111 sites at high 
ask for oil spills) They have also measured the travel cost c;; from location i to 
every potential spill site 7. Each spill site should be under the supervision of a single 
‘aspection office. 
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(a) Formulate a facilities location model to choose a minimum total cost collectior 
of offices and inspection assignments assuming that there is an annual fixed cost of” 
for operating any office. 

(b) Formulate a facilities location model to choose a minimum total cost collectio= 
of offices and inspection assignments assuming that fixed costs of operating office 
are unknown but that it has been decided to open at most 9. 


ae 


Modeling: We follow integer linear programming format 4 


(a) For this case the model required is 
14 111 14 


min Des yD Ci jXi,j +f a (total cost) 
l= J= — 


14 
s.t. > Seal jal lieaesesel TEL (each j inspected) 

111 
Yo xij Silly; i=1,...,14 — (éswitching) 
j= 
xij =Oorl i lipar 5 14a helt Le 
y; = Oorl dette see Lie! 


Since no capacities are specified for the offices, switching constraints use uj = 1 
Every spill site can be covered by any open office. Variables x;; are binary becaus 
spill sites must be inspected from a single office. | 


(b) For this case the model required is | 
14 111 


min ye SS GiiXay (travel cost) 
i=1 j=l 
14 

s.t. De xij =1 ey aay (each j inspected) 
i=1 


111 
s Mig = iy = lt (i switching) 


j=l 


14 
se yes (max nine chosen) 
i=l 

xij = Oor] j lg ered bet Lil 

yi =Oorl Fai 4 


Fixed charges have been omitted and a new constraint added that limits the num>= 
of offices to 9. 


Network Design Models 


Facility location models decide which nodes of a network to open. Network dessa 
or fixed-charge network flow models decide which arcs. A continuous network 
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in variables x;; is augmented with discrete variables y;; implementing fixed charges 
for opening/constructing/setting up arcs. 


Main constraints in x;,; mirror network flow models (Section 10.1) in conserving 


flow at each node. Switching constraints (definition | 1,3 |) turn on arc capacities u;, j 
if the fixed charge is paid. As usual, capacities must be derived from other constraints 
if not given explicitly—possibly by taking u;; = the largest feasible flow on arc (i, j). 


EXAMPLE 11.11: WASTEWATER NETWORK DESIGN 


Network design applications may involve telecommunications, electricity, water, gas, 
coal slurry, or any other substance that flows in a network. We illustrate with an ap- 
plication involving regional wastewater (sewer) networks.® 

As new areas develop around major cities, entire networks of collector sewers 
and treatment plants must be constructed to service growing population. Figure 11.8 
displays our particular (fictional) instance. 


7 Fixed Variable 
Are Cost Cost 
population (1,2) 240 21 
units (1,3) 350 30 
(2,3) 200 22 
(2, 4) 750 58 
(3,4) 610 43 
(3,9) 3800 1 
(4,3) 1840 49 
(4,8) 780 63 
(5,6) 620 44 
67) 800 51 
(6,7) 500 56 
(6, 8) 630 94 
(7,4) 1120 82 
(7,9) 3800 il 
(8,9) 2500 D 


FIGURE 11.8 Wastewater Network Design Example 


8Based on J. J. Jarvis, R. L. Rardin, V. E. Unger, R. W. Moore, and C. C. Schimpeler (1978), “Optimal 
Design of Regional Wastewater Systems: A Fixed Charge Network Flow Model,” Operations Research, 
26, 538-550. 
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Nodes 1 to 8 of the network represent population centers where smaller sewers feed 
into the main regional network, and locations where treatment plants might be built. 
Wastewater loads are roughly proportional to population, so the inflows indicated at 
nodes represent population units (in thousands). 

Arcs joining nodes 1 to 8 show possible routes for main collector sewers. Most 
follow the topology in gravity flow, but one pumped line (4, 3) is included. A large 
part of the construction cost for either type of line is fixed: right-of-way acquisition, 
trenching, and so on. Still, the cost of a line also grows with the number of population 
units carried, because greater flows imply larger-diameter pipes. The table in Figure 
11.8 shows the fixed and variable cost for each arc in thousand of dollars. 

Treatment plant costs actually occur at nodes—here nodes 3, 7, and 8. Figure 
11.8 illustrates, however, that such costs can be modeled on arcs by introducing an 
artificial “supersink” node 9. Costs shown for arcs (3, 9), (7,9), and (8, 9) capture 
the fixed and variable expense of plant construction as flows depart the network. 


Wastewater Network Design Example Model 


To place our wastewater network design problem in format] 11.31 |, we need capac- 


ities u;; on the arcs. None are explicitly provided, but it is not difficult to determine 
the maximum possible flow on each arc. For example, we may take 


u23 =27+3 = 30 


because no solution would send more than the 27 thousand population units at node 
1 and the 3 at node 2 along arc (2, 3). 

With such derived capacities, we may model the wastewater system design 
problem of Figure 11.8 as the following fixed-charge network flow problem: 


min 21x12 + 30x13 + 22x23 + 58x2,4 + 43x3,4 (total cost) 
+ 13,9 + 49x43 + 63x43 + 44x55 + 51x57 
+ 56x67 + 94x63 + 82x7,4 + 1x7,9 + 2x89 
+ 240y1.2 + 350y1,3 + 200y2,3 + 750y2,4 + 610y3,4 
+ 3800y3,9 + 1840y43 + 780y4.g + 6205.6 + 800ys.7 
+ 5006.7 + 630y6,3 + 1120y7,4 + 3800y7,9 + 2500y¢ 9 


St. —x1,2 — X13 =) (node 1) 
X1,2 — X2,3 — X2,4 = (node 2) (11.21) 
X1,3 + X2,3 + X43 — X34 — X39 = —14 (node 3) 
X24 +%3,4 +%7,4 — X43 —X4g = —36 (node 4) 
—%5,6 — X5,7 hell (node 5) 
X5,6 — X6,7 — X68 Sts (node 6) 
X5,7 + X6,7 — X7,4 — X7,9 = —13 (node 7) 
X4,8 + X6,3 — X8,9 = 0 (node 8) 


X3,9 + X79 + 8,9 = 122 (node 9) 
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0 £22 '= 27yr9,) OX x43 < 2713, 0 < x3 < 30yn5 (switching) 
O< x24 < 30y24, OS x34 <44y34, 0< x39 <122y39 

0 < x43 < 108y43, 0 <4 < 122y48, 0 < x56 <21ys56 

OS 45,7 <21y57, 0<%67 <29y67, 0 < x68 < 29V68 
O<x74<42y74, OS x79 <42y79, 0<xg9 <122y89 

Veo= Vor] all ares (i, j) 


Heavier lines in Figure 11.8 indicate an optimal design. Lines (1, 3), (2,3), 
(4, 3), (5, 7), and (6, 7) should be constructed, along with the plant at node 7. Total 
cost is $15,571,000. 


| SAMPLE EXERCISE 11.18: FORMULATING NETWORK DESIGN MODELS 


The following digraph shows possible routes for cable television lines from broad- 
casting center node 1 to towns at nodes 3 and 4. Node 2 is a connection box that may 
or may not be included in the ultimate system. Numbers on arcs are the fixed cost 
of the corresponding cable line. 

@) town 


10 
center @ 
town 


Formulate a fixed-charge network flow model to choose a least cost design providing 
service to both towns. 


Modeling: The problem provides no explicit supplies, demands, or capacities, and 
there are no variable costs. Thus we may assume that demand = 1 at both cities, with 
2 units of flow supplied at node 1. Capacity is then 2 on are (1,2), which might carry 
both flows, and 1 on all other arcs. 

Substituting these values in form produces the integer linear program- 
ming model 


min 10y1,2 + 25y1,3 + 33y1,4 + 8y2,3 + 6y2,4 (cost) 

St. —X1,2 — x43 —X1,4 = —2 (node 1) 
Mo — 93 — x4 3 = "0 (node 2) 
X1,3 + 2,3 a (node 3) 
X14 + X24 —_ ll (node 4) 


O<%12<2y12, O<M3<y13, O<x4<yi4 (switching) 
O<x23<yo3, O<x24<yo4 
Yi,2» V1.3 1,45 2,3, ¥2.4=O0orl 
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11.7 PROCESSOR SCHEDULING AND SEQUENCING MODELS 


Scheduling is the allocation of resources over time. The enormous range of applica- 
tions encompasses staffing activities like the ONB shift planning in Section 5.4, AA 
air crew scheduling in Section 11.3, Purdue final exam timetabling in Section 2.5, 
and construction project management in Section 9.7. In this section we introduce 
another very broad class: processor scheduling models that sequence a collection of 
jobs through a given set of processing devices. 


EXAMPLE 11.12: NIFTY NOTES SINGLE-MACHINE SCHEDULING 


We begin with the binder scheduling problem confronting a fictitious Nifty Notes 
copy shop. Just before the start of each semester, professors at the nearby university 
supply Nifty Notes with a single original of their class handouts, a projection of the 
class enrollment, and a due date by which copies should be available. Then the Nifty 
Notes staff must rush to print and bind the required number of copies before each 
class begins. 

During the busy period each semester, Nifty Notes operates its single binding 
station 24 hours per day. Table 11.10 shows process times, release times, and due 
dates for the jobs j = 1,..., 6 now pending at the binder: 


p; & estimated process time (in hours) job j will require to bind 


r; & release time (hour) at which job j has/will become available for processing 
(relative to time 0 = now) 


d; 4 due date (hour) by which job j should be completed 
(relative to time 0 = now) 


Notice that two jobs are already late (dj < 0). 


TABLE 11.10 Nifty Notes Scheduling Example 
Data 


Binder Job, j 
1 2 3 4 5 EG 
Process time, p; 12 8 3 10 Aa oeetiltc} 


Release time, 7; 20 -15 -12 -10 -3 2 


Due date, d; 10 2 1p -8 -6 60 


We wish to chose an optimal sequence in which to accomplish these jobs. No 
more than one can be in process at a time; and once started, a job must be completed 
before another can begin. 


Single-Processor Scheduling Problems 


Our Nifty Notes example is a very simple case of single-processor (or single-machine) 
scheduling. 


gee | Single-processor (or single-machine) scheduling problems seek an opti- _ 
mal sequence in which to complete a given collection of jobs on a single pro- _ 
cessor that can accommodate only one job at a time. 
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Usually (as in the Nifty Notes case), we also assume that preemption is not allowed. 
That is, one job cannot be interrupted to work on another. 


Time Decision Variables 


Since scheduling means assignment of resources over time, it is natural that one set 
of decision variables found in most models chooses job start or completion times. 


In our Nifty Notes example we employ 
x; A time binding starts for job j (relative to time 0 = now) 


Then one set of constraints requires each job to start after the later of now (time = 0) 
and the hour it will be released for processing: 


xj = max{0, 7} jSieee6 (earliest start) (11.22) 


We could just as well have modeled in terms of completion times. Still, both 
start and completion times need not be decided because one can be computed from 
the other through 


(start time) + (process time) = (completion time) 


(Recall that process times are given constants.) 


Conflict Constraints and Disjunctive Variables 


The central issue in processor scheduling is that only one job should be in progress 
on any processor at any time. If we know just start (or finish) times, it is not difficult 
to check for a violation or conflict. Still, it is not easy to write standard mathematical 
programming constraints that prevent conflicts. 

For any pair of jobs j and j’ that might conflict on a processor, the appropriate 
conflict constraint is either 


(start time of j) + (process time of j) < start time of j’ 
or (11.23) 
(start time of j’) + (process time of j’) < start time of j 


Either j must finish before j’ begins, or vice versa. But only one possibility can hold, 
and to determine which, we must know whether j or /’ starts first on the processor. 
Often, operations research analysts deal with conflict prevention (11.23) out- 
side the usual mathematical program format. Nonetheless, conflict avoidance re- 
quirements can be modeled explicitly with the aid of additional disjunctive variables. 


re OF aller each other 7 w ith 
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Then we can enforce conflict prevention (11.23) with pairs of linear constraints. 


1135 | A processor scheduling model with job start times x; and process times 
py can ee -_ between jobs] a“ 7 with — constraint = 


To illustrate for our Nifty Notes example, define 


i fe 1 if j binding comes before j’ 
i 


0 if j’ binding comes before j 


Construction | 11.35 | yields constraints 


xj + pj Sx +M1 — yj;) 


yf Se GO esi (11.24) 
Ay + Pj S xj + My; | 


As in many other cases, we consider only j’ > j, to avoid listing the same pair twice. 

To see how the constraints prevent conflicts, consider the specific case of j = 2 
j’ = 6. Using processing times from Table 11.10, the corresponding pair (11.24) is 

2 +8 <x6 +M(1 — yr6) 
x6 +18 < x2 + Myr6 

Ifjob 2 is started before job 6, y2,6 = 1, and the first constraint Keeps the start time for 
6-after the finish of 2. The second constraint is also enforced. However, the big-M 
term My26 makes it true for any x2, x. On the other hand, if job 6 is started before 


job 2, so that y26 = 0, the first constraint of the pair is discounted and the second 
enforced. 


Formulate integer linear programming constraints for fale schedules on a einai 
processor with jobs j = 1, ..., 3 having process times 14, 3, and 7, respectively. 


Modeling: We employ the decision variables 
x; & start time of job j 
rv ie if j is scheduled before j’ 
ies | 0 otherwise 


Then conflict constraints | 11.35 | assure that only one job is processed at a time are 


x =e 14 S20) SP M1 = V1.2) 
2 +3 <x, +My 
x, +14<%34+M0A- y1,3) 
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x3 +7 <x, + Myi3 
x2 +3 <x3+ M1 — yo3) 
x34+7 <x. + My23 


Variable-type restrictions 
X1,%X2, X3 >0 
Y1,2. Y1,3> 2,3 = O0orl 


complete the constraints required. 


Handling of Due Dates 


Readers will note that we have not formulated any constraints enforcing due dates 
for various jobs. This could easily be done by adding conditions of the form 


xt pj sqj 
However, it is not standard to enforce such requirements because there may 
very well be no feasible schedule that meets all due dates. In Nifty Notes data of Table 


11.10, for example, some due dates have already passed. it is far more customary to 
reflect due dates in the objective function as explained in the next subsection. 


xplicit constraints. Dates _ 


Processor Scheduling Objective Functions 


One of the intriguing features of scheduling models is the wide variety of objective 
functions that may be appropriate. 


i. 37 | Denoting the start time of jobj j=1,. wn by x;, the process time by pj, 
the release time ee ae the due date at a : , Disceegy scheduling objective 


oS Maximum completion time 
tea completion time _ 

Maximum flow time 

Bes flow time 


“nay + 


Total completion time, flow time, lateness, or tardiness may also be of interest. but 
optimization over each of these is equivalent to optimization over the correspondin= 
mean measure because the total is constant n times the mean. 
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The completion time measures in definition | 11.37 emphasize getting all jobs 
done as soon as possible. For example, the mean completion time version of our 
Nifty Notes example would have objective function 


min § [(x1 +12) + (xp +8) + (x3 +3) + (xg + 10) + (x5 +4) + (x6 + 18)] 


A corresponding optimal schedule has Nifty Notes binding jobs in sequence 3—5—2- 
4-1-6 with start times 


sf=25, = 7 aP=0, xf 15/5543) at 237 (11.25) 


and mean completion time 23.67. 

Completion time measures are particularly appropriate when there are a fixed 
number of jobs to complete and no more expected. Where the model relates to a 
more continuing operation, flow time may be more suitable. Flow time tracks the 
length of time that a job is in the system: 


flow time 4 (completion time) — (release time) 


The idea is to minimize work in process so that inventory costs for partially finished 
goods are reduced. 

The third category of measures becomes important when due dates are critical: 
Lateness counts both early and late jobs: 


lateness & (completion time) — (due date) 
Tardiness considers only the late ones (positive lateness): 
tardiness 4 max {0, (lateness) } 


For example, minimizing maximum lateness in the Nifty Notes case produces 
the objective function 


min max {(x; + 12 — 10), @ +8 — 2), (x3 +3 — 72), (v4 +10 +8), 
(x5 + 4+ 6), (x6 + 18 — 60)} 
The corresponding optimal schedule has Nifty Notes binding jobs in sequence 4—-2- 
5-3-1-6 with start times 
xi =22, 2 =14, a,=02, a, =O a,=10, xf = 34 (11.27) 


and maximum lateness (22 + 12 — 10) = 24 on job 1. Notice that this schedule 
differs significantly from the mean completion time schedule of (11.25). To decrease 
lateness, mean completion time has increased from 23.67 to 31.17. 


(11.26) 


The fellows table shows the process fies ieee times, due dates, and scheduled 


start times for three jobs. 


Job1 Job2 Job3 


Process time ils) 6 9 
Release time 5 10 0 
Due date 20 25 36 
Scheduled start 9 24 0 
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Compute the corresponding value of each of the eight objective functions in definition 


Analysis: Completion times (start + process) are 
9+15=24, 24+6=30, and0+9=9 


Thus maximum completion time is max{24,30,9} = 30 and mean completion 
time is 


(24 +3049) = 
Corresponding flow times (completion—release) are 
24.—3 = 19, -30=10= 20, “and .9-0=9 


Thus maximum flow time is max{19, 20, 9} = 20 and mean flow time is + (19+20+9) = 
16. 
Lateness of the three jobs (completion—due date) is 


24 —20=4, 30-25=5, and 9-—36=-27 


Thus maximum lateness is max{4, 5, -27} = 5 and mean lateness is 4 - 1 (445-27) = 
Finally, tardiness of the jobs (max{0, lateness}) is 


max{0,4}=4, max{0,5}=5, and max{0,—27} =0 


Thus maximum tardiness is max{4, 5, 0} = 5 and mean tardiness is 5(4 +5+0) =: 


ILP Formulation of Minmax Scheduling Objectives 


Disjunctive constraints of formulation | 11.35 |can be combined with any of the mean 


objective forms in list | 11.36 except tardiness to obtain an integer linear program- 
ming formulation of a processor scheduling problem. When tardiness or any of the 
minmax objectives are being optimized, however, the problem is an integer nonlinear 
program (INLP). 

We can convert any of these INLP forms to the more tractable ILP by using 


¢ min max objective of list [11.37] 11.37] can be linearized by intro- 
ducing a new deci ‘ision variable f to represent the objective function v value, then _ 


To illustrate, return to the Nifty Notes lateness objective of expression (11.26): 


min max {(x; +12 — 10), +8 — 2), (3 +3 —72), (x4 +10+8), 
(x5 +4 +6), (%6 + 18 — 60)} 
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We may convert to ILP form by introducing a new variable f and then solving 


min f 

Subs f >=x4.4+2 
f2=m+6 
f = x3 — 69 
f = xq 18 
f = x5 +10 
f = x6 —42 


(all original constraints) 


| SAMPLE EXERCISE I1.. 21: LINEARIZING SCHEDULING OBJECTIVES 


Using x; to represent the scheduled start time of each job j in Sample Exercise 11. 20, 
show how each of the following objective functions can be expressed in ILP format. 


(a) Maximum completion 


(b) Mean tardiness 
Modeling: We apply construction | 11.38). 


(a) To linearize maximum completion time, we introduce a new decision variable f 
and enforce new constraints to keep it as great as any completion time. Specifically, 
the formulation is 


min f 

st. f2xm4+h5 
i =ta-F6 
if ¥ee59 


(all original constraints) 
(b) To model tardiness, we introduce new decision variables t; > 0 for each job j and 
force it to be > lateness. Then the mean tardiness model is 


min 3(4 +f + +43) 
s.t. t > x, +15 —-20 


b > x2 +6-—25 
Bb > x3+9—-36 
Hi, 2,2 = 0 


(all original constraints) 


If job j is late, the corresponding new main constraint makes ¢; = lateness. If not, 
nonnegativity constraints 4 > 0 force tardiness = 0. 


Equivalences among Scheduling Objective Functions 


Different objective functions from list| 11.37 | do not always imply different optimal 
schedules. 


11.39 | The mean completion time, mean flow time, and mean lateness schedul- 
ing objective functions are equivalent in the sense that an optimal oT for 
one is also optimal for the others. . 
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11.40} An optimal schedule for the. maximum aS enes 
“optimal for. maximum tardiness. ee 


For example, the optimal mean completion time schedule (11.25) of our Nifty Notes 
example is also optimal for the mean flow time and mean lateness objective func- 
tions (principle| 11.39 }). Schedule (11.27), which minimized maximum lateness, also 
minimizes maximum tardiness (principle | 11.401). 


To see why mean completion time and mean flow time are equivalent, we need 
only rearrange defining sums: 


1 n 
mean flow time = — ' er 
= d& Di. 7G) 


= Lot) oT 


Le 
n 

= (mean completion time) — — I 

( p dais dX j 
Expressed this way, it is apparent that the objective functions differ only by the con- 
stant last term. Adding or subtracting such a constant to the objective function cannot 
change what solutions are optimal. Similar arguments equate mean completion time 

and mean lateness. 

Connection} 11.40 | between maximum lateness and maximum tardiness is also 


straightforward. If at least one job must be late in every schedule, maximum lateness 
= maximum tardiness. If no job has to be late, all schedules are optimal for the 
maximum tardiness objective, including any optimal for maximum lateness. 


Job Shop Scheduling 


accomm: ce only 6 ‘one job at time, 


EXAMPLE 11.13: CUSTOM METALWORKING JOB SHOP 


We illustrate job shop scheduling with a fictitious Custom Metalworking company 
which fabricates prototype metal parts for a nearby engine manufacturer. Figure 
11.9 provides details on the 3 jobs waiting to be scheduled. First is a die requiring 
work ona sequence of 5 workstations: 1 (forging), then 2 (machining), then 3 (grind- 
ing), then 4 (polishing), and finally, 6 (electric discharge cutting). Job 2 is a cam shaft 
requiring 4 stations, and job 3 a fuel injector requiring 5 steps. Numbers in boxes 
indicate process times 


Pix & process time (in minutes) of job j on processor k 


For example, job 1 requires 45 minutes at polishing workstation 4. 
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k Workstations 

die 1 Forging 
2 Machining center 
3 Grinding 
4 Polishing 
5 Drillin 

2. Camshaft : oo 

sai 6 = Electric discharge 

7 Heat treatment 


3. Fuel Injector 


FIGURE 11.9 Custom Metalworking Example Jobs 


Any of the objective function forms in list could be appropriate for 
Custom Metalworking’s scheduling. We will assume that the company wants to 
complete all 3 jobs as soon as possible (minimize maximum completion), so that 
workers can leave for a holiday. 


Custom Metalworking Example Decision Variables and Objective 


Job shop scheduling involves deciding when to start each step of each job on its 


processor. Thus start time decision variables 11.33 | are now indexed by both job 
and processor: 


Xx & start time of job j on processor k 
Our assumed makespan scheduling objective can then be expressed as 
min max{x,6 +1, x23 +6, %3,4 + 25} 
Notice that only the last step of each job is reflected. Completion of a multiprocessor 
job means completion of all steps. 
Precedence Constraints 


Steps of the various jobs being scheduled in a job shop must take place in the sequence 
given. That is, start times are subject to precedence constraints. 


11.42 7 The eS cee eee that jobj £ ‘must complete on processor k be- 


Job shop models have precedence constraints} 11.42 | between each step and its 
successor in each job. For instance, job 1 in Figure 11.9 implies precedence constraints 
Mra +3 X12 
x412+10< x13 
M1348 < x14 
X14 +45 < x16 
to maintain the required processing sequence. 
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SAMPLE EXERCISE 11.22: FORMUL i G Jop SHOP ERECEDEN 


A job shop must schedule product t which requires 12 minutes on nen cee 1 followed 
by 30 minutes on machine 2, and product 2, which requires 17 minutes on machine 1 
followed by 29 minutes on machine 3. Formulate the implied precedence constraints 
in terms of the decision variables 


Xx & start time of product j on machine k 


Modeling: There is one ie constraint for each job because each has only 


two steps. In accord with| 11. 41.42 |, those constraints are 


x11 +12 < x12 


X21 +17 < x3 


Conflict Constraints in Job Shops 


As in one machine case such as Nifty Notes, job shop models must also confront 
the possibility of conflicts—jobs scheduled simultaneously on the same processor. 
For example, in the Custom Metalworking example of Figure 11.9, jobs 1 and 2 may 
conflict at workstation 1, which both require. One must complete before the other 
can begin. 

Paralleling | 11.34) and} 1 , we may model conflicts by introducing the new 
discrete decision variables 


Pal ie! if j is scheduled before job j’ on processor k 
Ve : 
0 otherwise 


tL. 43 Job oe SS can prevent conflicts between = by Sue ROW: 


Xj at Py & Se + My a 


for each ‘j. j' that both pie any processor k. ee x;,~ denotes the start time _ 
ti 


For instance, the possible conflict between Custom Metalworking jobs 1 and 2 at 
workstation 1 produces constraint pair 


x1 +6 <¥214+M1 —- yi2,1) 
X21 +3 < 4114+ Myi21 


If job 1 uses the processor first, v1.2.1 = 1, and the first constraint is enforced. If job 
2 comes first, y1,2,1 = 0, and the second constraint controls. 
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SAMPLE EXERCISE 11.23; FORMULATING JOB SHOP CONFLICT CONSTRAINTS 
Return to the job shop of Sample Exercise 11.22 and formulate all constraints re- 
quired to prevent conflicts. 

Modeling: Conflicts can occur only on machine 1, which is the only one required for 
both products. Thus we require only one binary variable, 


1 if product 1 is first on machine 1 


A 
%121=) ( if product 2 is first on machine 1 


and the needed constraints | : are 


Xy14+12 <x21+M0 —yi21) 
X24 +17 <x11+ Myi 21 


Here M = 12 +17 = 29 would be large enough to have the desired effect of dis- 
counting whichever constraint should not really apply. 


Custom Metalworking Example Model 
Combining our maximum completion time objective function with all required prece- 
dence and conflict constraints produces the following complete model of the Custom 
Metalworking example in Figure 11.9: 
min max{x1,6 +1, x23 +6, x34 + 25} (maximum completion) 
Sti | Spa US aco, (job 1 precedence) 
M2+10 = x13 
MeP8 2x14 
M14+45 < x16 
x27 +50 < x21 (job 2 precedence) 
x21+6 <%22 
x22 +11 < 2x23 
139 te ie 3.4) (job 3 precedence) 
a3 +9 = %3,5 
%35+2 <%36 (11.28) 
X36+1 < x34 
41+6 <x21+M0—y121) (workstation 1 conflicts) 
Rt Po Se eye 
X42 +10 <x22+ M1 —y122) (workstation 2 conflicts) 
X22 +11 <x12+Myi22 
x12 +10 S %o FM —yia9) 
%32+5 <%12+ Myr 32 
#9 +11 = 9+ MA = 933) 
%32+5 <%22+ Myr32 
m134+8 <%3+M0—yi23) (workstation 3 conflicts) 


x23 +6 
x13 +8 
X33 + 9 
x23 +6 
x33+9 
4445 <x34+ M1 — yi3,4) 
X34425 <x14+ Myi34 


< x13 + Myi23 
< x33 + M(1 — y133) 
< %1,3 + Myi 33 
ota 5 CI V738) 
< x23 + My233 


X6+1 <x36+M(1 — y136) 
x3641 <x16+ Myi 3.6 
all xjn > 0 


all yjj,4 =Oorl 


An optimal solution uses start times 


Pie hin ipibet & eee 
xt, = De Xt. = op x13 = 15, 

alee rian fig Mes 
37> 0, 34= 50, X39 = 56, 

eearecl + le rie 
35 = 0, %33= 5f X35 = 14, 


to complete all jobs in 88 minutes. 


EXERCISES 


11-1 A fertilizer plant can make a product by any of 
3 processes. Using decision variables x; 4 number 
of units produced by process j, the following linear 
program computes a minimum cost way to produce 
150 units with available resources: 

15x,+ llx2 + 18x3 

x1 +X2 +x3 = 150 

2x, + 4x2 + 2x3 < 310 

4x, + 3x2 +x3 < 450 

Xj, X2,%3 = 0 


min 
s.t. 


(a) [| Explain why this LP implicitly assumes 
that objective function coefficients are variable 
costs. 

[9] ©] Use class optimization software to solve 
the given LP. 

[6] Formulate a revised ILP model implement- 
ing a requirement that only one of the 3 activities 
may be used. 

[>] &] Use class optimization software to solve 
the ILP of part (c). 

[] Formulate a different revised ILP imple- 
menting a fixed setup cost of 400 charged for 
each activity used at all. 

[6] © Use class optimization software to solve 
the ILP of part (e). 


(b) 
(c) 


(d) 
(e) 


(f) 
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(workstation 4 conflicts) 


(workstation 6 conflicts) 


Xi4= 42, x16 =87 
X53 = 67 


xi,=16, x4,=17 


(g) Formulate another revised ILP implement- 
ing a requirement that an activity can be used 
only if a minimum of 50 units are produced. 

(h) Use class optimization software to solve 
the ILP of part (g). 


11-2 A computer distributor can purchase worksta- 
tions from any of 3 suppliers. Using decision vari- 
ables x; 4 number of units purchased from supplier 
j, the following linear program computes a minimum 
cost way to purchase 300 workstations within appli- 
cable limits: 


min 5x, + 7x2 + 6.5x3 
s.t. FX. +%3 = 300 
3x, + 5x2 + 4x3 < 1500 
0 <x, < 200 
0 < x2 < 300 
0 < x3 < 200 


(a) through (h) asin Exercise 11-1 using 2 ss°o9 


of 100 and a minimum purchase of += 


11-3 A retired executive has up to $8 m 
ing table shows the purchase price anc ™ 

10-year return (in millions of dollars) a “t= + ™s — 
ings that he is considering. 


hh 
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Building 
1 2 3 4 


Price AOL, 3:8. 16:0) . 72 
Return 45 41 80 7.0 


The executive wishes to choose investments that 
maximize his total return. Assume that every op- 
tion is available only on an all-or-nothing basis. 


(a) [Q] Formulate a knapsack ILP to choose an op- 
timal investment plan. 
(b) [§] Solve your knapsack by inspection. 


11-4 The River City redevelopment authority wants 
to add a minimum of onc thousand new parking 
spaces in the downtown area. The following table 
shows the estimated cost (in millions of dollars) of 
the 4 proposed projects and the number of spaces 
each would yield (in hundreds). 


Project 
2s 4 
Cost HOP Os Aye ne 


Spaces 8 58 6 6 


The authority wants to meet its goal at minimum to- 
tal cost. Assume that every project is available only 
on an all-or-nothing basis. 


(a) Formulate a knapsack ILP to choose an optimal 
parking program. 
(b) Solve your knapsack by inspection. 


11-5 Silo State’s School of Engineering is preparing 
a 5-year plan for building construction and expan- 
sion to accommodate new offices, laboratories, and 
classrooms. The Electrical Engineering faculty has 
proposed projects for all 3 available parcels of land: 
a digital circuits lab on the northwest parcel at $48 
million, a faculty office annex on the southeast par- 
cel at $20.8 million, and a computer vision lab on the 
northeast parcel at $32 million. The Mechanical En- 
gineering faculty has 3 alternative proposals for the 
same northwest parcel: a large lecture room build- 
ing at $28 million, a heat transfer lab at $44 million, 
and a computer-aided design expansion at $17.2 mil- 
lion. The Industrial Engineering faculty has only 2 
proposals: a manufacturing research center on the 
southeast parcel at $36.8 million, and a tunnel from 
their current building to the new center for an addi- 
tional $1.2 million. The Dean of Engineering scores 
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the impact of these projects as 9,2, and 10 for the EE 
proposals, 2, 5, and 8 for the ME alternatives, 10 and 
1 for the IE ideas. He wishes to allocate his available 
$100 million to maximize the total impact, selecting 
projects on an all-or-nothing basis with at most one 
per land parcel. 


(a) Describe verbally the budget limits, mutual 
exclusiveness constraints, and project depen- 
dency requirements of this capital budgeting 
problem. 

(b) [| Formulate a capital budgeting ILP to select 
an optimal combination of proposals. 

(c) [|] & Use class optimization software to solve 
your ILP model. 


11-6 A small pharmaceutical research laboratory 
must decide which product research activities to un- 
dertake with its $25 million available in each of the 
next two 5-year periods. The products optamine and 
feasibine are ready for field testing in the first 5-year 
period at a cost of $13 million and $14 million, re- 
spectively. Discretol and zeronex are at an earlier, 
development stage. Proposed development activi- 
ties would require $4 million in the first 5 years for 
discretol and $3 million in the second. Correspond- 
ing values for zeronex are $2 million and $6 million. 
Field testing of the two products may also be cho- 
sen in the second 5-year period for $10 million and 
$15 million, respectively, if the corresponding devel- 
opment activity was undertaken in the first 5 years. 
The company wishes to maximize future profits from 
field-tested products, which it estimates at $510 mil- 
lion for optamine, $640 million for feasibine, $580 
million for discretol, and $469 million for zeronex. 
All projects must be adopted on an all-or-nothing 
basis, and no more than one of the two development 
projects may be selected. 


(a) through (c) as in Exercise 11-5. 


11-7 The map that follows shows the locations of § 
applicants for low-power radio station licences and 
the approximate range of their signals. 


Regulators have scored the quality of applications on 
a scale of 0 to 100 as 45, 30, 84, 73, 80, 70, 61, and 91, 
respectively. They wish to select the highest-quality 
combination of applications that has no overlap in 
signal ranges. 


(a) [§] Formulate this problem as a set packing ILP. 
(b) [6] & Use class optimization software to solve 
your ILP. 


11-8 Time Sink Incorporated is about to begin selling 
its computer game software to college students in the 
midwest. The following table shows the states that 
could be covered by salespersons based at 4 possible 
locations, together with the annual sales (in thou- 
sands of dollars) expected if all those states were 
covered from that base. 


Base 
Ames Beloit Normal Avon 

MN x x = mes 
TA x x x — 
MO x os x — 
WI — x x — 
IN — — x x 
KY —_— — — x 
Sales 11S 90 150 126 


Time Sink wants to choose a collection of bases that 
maximizes total sales without assigning the same 
state to more than one base. 


(a) and (b) as in Exercise 11-7. 


11-9 The following map shows the 8 intersections 
at which automatic traffic monitoring devices might 
be installed. A station at any particular node can 
monitor all the road links meeting that intersection. 
Numbers next to nodes reflect the monthly cost (in 
thousands of dollars) of operating a station at that 
location. 


40 
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(a) [9] Formulate the problem of providing full co 
erage at minimum total cost as a set covering 
ILP. 

(b) [| & Use class optimization software to solve 
your ILP of part (a). 

(c) Revise your formulation of part (a) to obtain an 
ILP minimizing the number of uncovered road 
links while using at most 2 stations. 

(d) &| & Use class optimization software to solve 
your ILP of part (c). 


11-10 Top Tool Company wishes to hire part-time 
models to pass out literature about its machine 
tools at an upcoming trade show. Each of the 6 
available models can work 2 of the 5 show days: 
Monday-—Tuesday, Monday—Wednesday, Monday— 
Friday, Tuesday—Wednesday, Tuesday—Friday, and 
Thursday-—Friday. If there is no more than one day 
separating their duty days, models will be paid $300. 
but the charge is $500 if two or more days intervene. 
Top Tool seeks a minimum cost way to have a least 
one model on all 5 days. 


(a) through (d) as in Exercise 11-9 with the revised 
model minimizing the number of days uncov- 
ered using just 2 models. 


11-11 Air Anton is asmall commuter airline running 
6 flights per day from New York City to surround- 
ing resort areas. Flight crews are all based in New 
York, working flights to various locations and then 
returning on the next flight home. Taking into ac- 
count complex work rules and pay incentives, Air 
Anton schedulers have constructed the 8 possible 
work patterns detailed in the following table. Each 
row of the table marks the flights that could be cov- 
ered in a particular pattern and the daily cost per 
crew (in thousands of dollars). 


Work Flight 
Pattern 1 2 3 4 5 6 Cost 
1 x x 1.40 
2 — oS Si 0.96 
3) — = — x x — 1.52 
4 x — — xX x 1.60 
5 YS hoy eis x 32 
6 x x 1.12 
f —- —- — x — x 0.84 
8 — x Mee 54 


The company wants to choose a minimum total cost 
collection of work patterns that covers all flights ex- 
actly once. 
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(a) &] Formulate this problem as a set partitioning 
ILP. 

(b) [S| & Use class optimization software to solve 
your ILP. 


11-12 A special court commission appointed to re- 
solve a bitter fight over legislative redistricting has 
proposed 6 combinations of the 5 disputed counties 
that could form new districts. The following table 
marks the counties composing each proposed district 
and shows the district’s deviation from the equal- 
population norm. 


District 
County 1 2 3 4 5 6 


1 x —- — DS x 
2 x — x —- =— x 
3} = Cl x x x _ 
4 — x — Be —_ =— 
5) _— x —- —- — 
Deviation 0.5 0:5 0.6.13 0.7 1.2 


The court wants to select a minimum total deviation 
collection of districts that includes each county ex- 
actly once. 


(a) and (b) as in Exercise 11-11. 


11-13, Mogul Motors is planning a major overhaul 
of its automobile assembly plants as it introduces 
4 new models. Exactly one exisiting plant must be 
converted to assemble each model. The following 
table shows for each model and plant the cost (in 
millions of dollars) of modifications at the plant to 
produce the model. Those marked with a dash reflect 
plants not large enough to accommodate the needed 
activity. 


Plant 
Model 1 2 3 4 
1 18 26 — 31 
2 — 50 22 — 
3 40, 429.6 352) 2739. 
4 — — 4 46 


Mogul seek a minimum total cost way to make the 
conversion. 


(a) [| Formulate this problem as a linear assign- 
ment LP. 

(b) [§] Explain why a binary optimal solution is 
guaranteed even though your model is a linear 
program. 
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(c) [| & Use class optimization software to solv: 
your assignment LP. 


11-14 The sister communities program pairs citic 
in Russia with cities in the United States that hav« 
a similar size and economic base. Visits are thes 
exchanged between the sister communities to i= 
prove international understanding. The followin: 
table shows the program’s compatibility scores (0+ 
100) for the 4 U.S. and 4 Russian cities about to j 
the program. 


Russian 


BRWNR 
Re} 
is) 
a 
n 
Nn 
Go 
Nn 
No} 


Sister communities seeks a maximum total comp: 
ibility pairing. 
(a) through (c) as in Exercise 11-13. 


11-15 Sandbox State University is rearranging © 
locations of 3 equal-sized academic departments 
provide for better faculty communication. The * 
lowing tables shows the estimated number of pers 
to-person contacts per month between members 
the various faculties, and the distances (in thousas~ 
of feet) between available office locations. 


Interaction 
English Math 
History 20 12 
English — 14 
Distance 
2S: 
1 Sa) 
2 — 1 


Sandbox State wants to place one department in cas 
location in a way that minimizes total distance tr 
eled for faculty interactions. 


(a) [Q] Formulate this problem as a quadratic = 
signment INLP. 

(b) [| Explain why the objective function in 
problem must be quadratic rather than linc 

(c) [6] Compute an optimal assignment in » 
INLP by inspection. 


11-16 The River City Operations Research society 
is planning a meeting that will have 2 morning and 
2 afternoon sessions running at the same time. The 
4 sessions will be on LP, NLP, ILP, and INLP, re- 
spectively, but times have not yet been fixed. The 
following table shows the estimated number of at- 
tendees who would like to be able to attend both of 
each combination of sessions. 


NLP ILP INLP 
LP 10 30 14 
NLP _ 5 8 


IEP _ — 18 


The society would like to arrange sessions to min- 
imize the number of persons who cannot attend a 
desired pair of sessions because they occur simulta- 
neously. 


(a) through (c) as in Exercise 11-15. 


11-17 A warehouse facility has packing stations at 
both its front and back entrances. The following ta- 
ble shows the number of ton-feet (in thousands) of 
materials handling that would be required to move 
each of the 6 pending jobs to either of the 2 stations, 
along with the number of hours packing that would 
be required at whichever station does the work. 


Job 
1 2 3 4 5 6 


Front (21) (17 10M S30 140) 822 
Backes sl Si glSaimZonn 24559332 29 


Time 44 60 51 80 73 67 


Schedulers seek a minimum handling plan that com- 
pletes all packing within the 200 hours available at 
the front-station and 190 hours available at the back. 
Assume that jobs must go entirely to a single packing 
station. 


(a) [Q] Formulate this problem as a generalized as- 
signment ILP. 

(b) [Q] Explain why this is a generalized rather than 
an ordinary assignment problem. 

(c) [6] © Use class optimization software to solve 
your ILP. 


11-18 Three professional baseball teams are trying 
to find places for 6 available players within their re- 
maining salary limits of $35 million, $20 million, and 
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$26 million, respectively. The following table shows 
how valuable each player would be to each team on 
a scale of 0 to 10, and the player’s current annual 
salary (in millions of dollars). 


Value 
Player 1 2 3. Salary 
1 Sk Gt 910 10 
2 Vaetey oie 6) 13 
3 Stee io) 8 
4 G7 3 3 6 
5 8) etal 6 15 
6 Oe 97 10) 22, 


We want to find a maximum total score allocation of 
players to teams that fits with in salary limits. 


(a) through (c) as in Exercise 11-17. 


11-19 Military commanders are planning the com- 
mand structure for 6 new radar stations. Three 
commanders will each be in charge of two of the 
stations. The following table shows the projected 
cost (in millions of dollars) of building the necessary 
communication links to connect jointly commanded 
locations. 


I 40s 65° 208 ail 55 
Qo east © 2075397" "40" 21 
SSS) OSS] 6815519422 
4 — — — 30 39 
5 -—- -—- — — 47 


Planners seek a minimum cost way to organize the 
command. 


(a) [] Formulate this problem as a matching ILP. 

(b) [6] Explain why this is not an assignment prob- 
lem. 

(c) &] & Use class optimization software to solve 
your ILP. - 


11-20 Awesome Advertising manages the televis: 
promotion of a variety of products. In the 
months they are planning to cross-adverti 
their items by running interlocking television & 
that mention both products. The followne 
shows Awesome’s estimate of the number 

(in millions) who might be interested join" 

pair of products. 
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Awesome wants to find a product pairing that max- 
imizes the appeal, with each product in exactly one 
pair. 


(a) through (c) as in Exercise 11-19. 


11-21 Engineers are designing a fixed route to be fol- 
lowed by automatic guided vehicles in a large manu- 
facturing plant. The following table shows the east— 
west and north-south coordinates of the 6 stations to 
be served by vehicles moving continuously around 
the same route. 


1 2 3 4 5 6 


E/W 20 40 180 130 160 50 
N/S 90 S701. P20E9) 40 S02 80 


Since traffic must move along east-west or north— 
south aisles, designers seek a route of shortest total 
rectilinear length (see Section 4.6). 


(a) [| Explain why this problem can be viewed as 
a traveling salesman problem. 

(b) [| Explain why distances in this problem are 
symmetric, and compute a matrix of rectilinear 
distances between all pairs of points. 

(c) [| Formulate this problem (incompletely) as an 
ILP with main constraints requiring only that 
every point be touched by 2 links of the route. 

(d) [9] & Use class optimization software to show 
that your ILP of part (c) produces a subtour 1- 
2-6-1. 

(e) [| Formulate a subtour elimination constraint 
that precludes the solution of part (d). 

(f) &] & Use class optimization software to show 
that an optimal route results when your subtour 
elimination constraint is added to the formula- 
tion of part (c). 

(g) [| Formulate this problem as a quadratic as- 
signment INLP. 


11-22 An oil company currently has 5 platforms 
drilling off the Gulf coast of the United States. The 
following table shows the east-west and north-south 
coordinates of their shore base at point 0 and all the 
platform locations. 
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0 1 2 3 4 5 


E/W. = 3808= 10 +607 230'S585 = Sits 
N/S 95h) rlisvn F0nr L0oeI5= 280) 


Each day a helicopter delivers supplies by flying f= 
the base to all platforms and then returning to bas: 
Supervisors seek the route of shortest total lengt= 


(a) through (f) as in Exercise 11-21 using strais™ 
line (Euclidean) distance and subtour 0-2— 


11-23 Every week Mighty Mo Manufacturing mas — 
one production run of each of its 4 different kings — 
metal cookware. Setup times to make any parts) 
lar product vary depending on what was produc.” 
most recently. The following table shows the \= 
(in hours) required to convert from any produc ~ 
any other. 


1 2 3 4 
— 42 15 65 


1 

Dine Ope a ihe og () 
Sie lee eae 28.0) 
4 55 18 60 — 


Mighty Mo would like to find the production ~ 
quence that minimizes total setup time. 


(a) [§] Explain why this problem can be viewes — 
an asymmetric traveling salesman problem 

(b) [Q] Formulate this problem (incompletely =~ 
linear assignment problem to choose a sus 
sor for each product. 

(c) [| & Use class optimization software to =~ 
that your ILP of part (b) produces a subtow 
3-1. 

(d) [§] Formulate a subtour elimination cons. 
that precludes the solution of part (c). 

(e) [O] & Use class optimization software to <> 
that an optimal route results when your su 
elimination constraint is added to the form 
tion of part (b). 

(f) [€] Formulate this problem as a quadrat: 
signment INLP. 


11-24 Every weekday afternoon, at the hes 
the rush hour, a bank messenger drives from 
bank’s central office to its 3 branches and ro 
with noncash records of the day’s activity. To. ~ 
lowing figure shows the freeway routes that =~ 
hopelessly clogged by traffic at that hour and 
timated driving time for each (in minutes). 


10 


45 25 


15 


The bank would like to find a route that minimizes 
total travel time. 


(a) through (e) as in Exercise 11-23 with subtour 
0-3-0. 


11-25 Gotit Grocery Company is considering 3 lo- 
cations for new distribution centers to serve it cus- 
tomers in 4 nearby cities. The following table shows 
the fixed cost (in millions of dollars) of opening each 
potential center, the number (in thousands) of truck- 
loads forecasted to be demanded at each city over the 
next 5 years, and the transportation cost (in millions 
of dollars) per thousand truckloads moved from each 
center location to each city. 


Fixed oe 
Center Cost 1 2 3 4 
1 200 6 5 9 3 
> 400 4 3 5 6 
3 225 5 8 D, 4 
Demand — LIST Seen St $95 


Gotit seeks a minimum cost distribution system as- 
suming any distribution center can meet any or all 
demands. 


‘a) [Q] Formulate this problem as a facilities loca- 
tion ILP. 

(b) | & Use class optimization software to solve 
your ILP. 


11-26 Basic Box Company is considering 5 new box 
designs of different sizes to package 4 upcoming lines 
of computer monitors. The following table shows 
the wasted space that each box would have if used 
to package each monitor. Missing values indicate a 
Dox that cannot be used for a particular monitor. 
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Monitor 
Box 1 2 3 4 
1 =. 
2 20S 25 
3 40 — 40 30 
4 — 10 7 — 
5 — 40 8:0 — 


Basic wants to choose the smallest number of box 
designs needed to pack all products and to decide 
which box design to use for each monitor, to mini- 
mize waste. 


(a) and (b) as in Exercise 11-25. (Hint: Use a large 
positive constant for fixed charges.) 


11-27 The figure that follows shows 5 pipelines un- 
der consideration by a natural gas company to move 
gas from its 2 fields to its 2 storage areas. The num- 
bers on the arcs show the number of miles of line that 
would have to be constructed at $100,000 per mile. 


The figure also shows that storage facilities are both 
already connected to the company’s main termina 
through existing lines. An estimated 800 million cu- 
bic feet must be shipped each year from field 1 * 
the terminal, and 600 million from field 2. Variab' 
shipping cost is $2000 per million cubic feet on 222° 
link of the network, and all links have an annual < 
pacity of 1 billion cubic feet. The company wan: 
minimum total annual cost system for the requires 
shipping. 
(a) [Q] Formulate this problem as a network ¢ 

TEP. 


(b) [Q] 2 Use class optimization software ° 
you ILP. 


11-28 Dandy Diesel manufacturing somos 
sembles diesel engines for heavy construction sue 
ment. Over the next 4 quarters th 2 
to ship 40, 20, 60, and 15 units. respect 
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more than 50 can be assembled in any quarter. There 
is a fixed cost of $2000 each time the line is setup for 
production, plus $200 per unit assembled. Engines 
may be held over in inventory at the plant for $100 
per unit per month. Dandy seeks a minimum total 
cost production plan for the 4 quarters, assuming that 
there in no beginning or ending inventory. 


(a) and (b) as in Exercise 11-27. (Hint: Create 
nodes for cach quarter and a common source 
node for production arcs.) 


11-29 Top-T shirt company imprints T-shirts with 
cartoons and celebrity photographs. For each of 
their 4 pending contracts, the following table shows 
the number of days of production required, the ear- 
liest day the order can begin, and the day the order 
is due. 


de 22828 Savi ed 


Production 10 316 8 
Earliest OF 20) Ais 1D 
Due Date 126 SOP 20ia 21 


The company wants to design an optimal schedule 
assuming that contracts can be processed in any se- 
quence but that production cannot be interrupted 
once a job has started. 


(a) [Q] Ignoring objective functions for the mo- 
ment, formulate constraints of a single-machine 
ILP to select an optimal start time for each con- 
tract. 

(b) [| Evaluate each of the 8 objective function 


in principle | 11.37 | for the schedule with start 
times 2, 20, 23, and 12 for the four contracts, 


respectively. 

(c) [] Extend your constraints of part (a) to formu- 
late an ILP to compute a minimum mean com- 
pletion time schedule. 

(d) [§] & Use class optimization software to solve 
your ILP of part (c). 

(e) [S| Without actually solving, list the other ob- 


jective functions of | 11.37 | for which your 
schedule of part (d) must be optimal. 

(f) [Q] Extend your constraints of part (a) to formu- 
late an ILP to compute a minimum maximum 
lateness schedule. 

(g) | & Use class optimization software to solve 
your ILP of part (f). 

(hb) [&] Without actually solving, list the other ob- 


jective functions of | 11.37; for which your 


schedule of part (g) must be optimal. 
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11-30 Sarah is a graduate student who must mas 
large experimental runs on her personal comput= = 
part of her thesis research. The jobs require virtas 
all the computer’s resources, so only one can be = 
cessed at a time and none can be interrupted ons ~ 
has begun. The following table shows the numb= ~ 
days of computing each job will require, the ear 
that all data will be available, and the day Sara= ~~ 
promised the result to her thesis advisor. 


Lee 23 4 


Time 15 8 20 6 . 
Earliest 0 OF 104 a0 
Promise 20 20 30 20 


Before beginning any work, Sarah wants to coms 
an optimal schedule. Assume promised times — 
only targets. 


(a) through (h) as in Exercise 11-29, evaluating 
schedule with start times 8, 0,23, and 43. ress 
tively. 


11-31 Three new jobs have just arrived at Fang © 
ishing’s main furniture restoration shop. The i> 
ing table shows the sequence that each must 5 
through the company’s 3 finish removal procs 
and the time required for each. 


Process Time 
Job Sequence thas 8) 
il 1-2-3 10 


3 
2 1-3-2 eae 1 
3 2-1-3 12 6 


Once a process is begun, it cannot be interros 
Although the shop was empty when the new jo™ — 
rived, Fancy expects more in the next few days 
maintain efficiency, they seek a schedule that => 
mizes the average time that a job is in the shor 


(a) [Q] Ignoring objective functions for the = 
ment, formulate constraints of a job shoe © 
to select an optimal start time for each > 
each machine. 

(b) [] Which of the 8 objective functionsin — 
is appropriate for this problem? 

(c) [|] Complete an ILP model by introduciss 
objective. 

(d) [S| & Use class optimization software to = 
your ILP model for an optimal schedule. 


11-32 A team of auditors has divided itself into 3 
groups, each to examine one category of records. 
Each group will review their speciality area for all 
3 subsidiaries of the client being audited, but the re- 
guired sequence and times differ, as shown in the 
following table. 


———_—<—< 


Group Time 

Subsidiary Sequence 1 2 3 
ih 1-3-2 4 Sl? 

2 2-1-3 Gm sr "3 

3 3-2-1 oN eT 3 


Once a group starts on a subsidiary, it should finish 
all work either before it moves to another or before 
a different group begins on theirs. The team seeks 
a schedule that will complete all work at the earliest 
possible time. 


a) through (d) as in Exercise 11-31. 


11-33 With the addition of a new plant, Monsanto? 
now has more capacity than it needs to manufac- 
ture its main chemical product. Numerous reactors 
i=1,...,mcan be operated ata variety; =1,...,n 
of discrete combinations of settings for feed rate, re- 
actor velocity, and reactor pressure. Both the pro- 
duction yield p;; and the operating cost c; j Vary with 
reactor and setting. Formulate an ILP model to find 
the least cost way to fulfill total production target 
5 in terms of the decision variables (Cicsilbales si7ae 
| = eee) 


1 if reactor i operates at setting j 


A 
Xi,j — 


0 otherwise 


11-34 W.R. Grace” strip mines phosphates in strata 
aumbered from = 1 at the top toi = n at the deep- 
est level. Each stratum must be removed before the 
next can be mined, but only some of the layers con- 
‘ain enough suitable minerals to justify processing 
‘ato the company’s three products: pebble, concen- 
trate, and flotation feed (j = 1, 2,3). The company 
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can estimate from drill samples the quantity a 
product j available in cach stratum i, the fraction 

of BPL (a measure of phosphate content) in the p= 
of i suitable for j, and the corresponding fraction ; 
of pollutant chemicals. They wish to choose a mininz 
plan that maximizes the product output while keep- 
ing the average fraction BPL of material processec 
for each product j at least b; and the average pol- 
lution fraction at most p;. Formulate an ILP model 
of this mining problem using the decision variables 
Gaa5 an) 


if stratum i is removed 


otherwise 


if stratum i is processed 


otherwise 


11-35 Ault Food Limited" is planning the pro- 
duction and distribution system for its new line of 
food products. Plants may be opened at any of 
sites i = 1,...,7, and warehouses at locations j= 
1,..., 13, to meet demands d, at customer regions 
k = 1,...,219. Each plant costs $50 million to 
open and produces up to 30 thousand cases per year. 
Warehouses cost $12 million to open and handle up 
to 10 thousand cases per year. Transportation costs 
are 7; per case for rail shipment from plant i to ware- 
house j, and ;, per case for trucking from warehouse 
j to customer k. No direct shipments from the plants 
are allowed. Formulate ILP model to decide which 
facilities to open and how to service customers using 
the decision variables (i = 1,...,7; 7 = 1,..., 13: 
ee a 219) 


Xi, 2 thousand of cases produced at plant i and 
shipped to customer k via warehouse j 


ir rail if plant i is opened 
nk 0 otherwise 
1 if warehouse j is opened 
ws 
0 otherwise 


Based on R. R. Boykin (1985), “Optimizing Chemical Production at Monsanto,” Interfaces, 15:1, 


53-95. 


'0Based on D. Klingman and N. Phillips (1988), “Integer Programming for Optimal Phosphate- 
Mining Strategies,” Journal of the Operational Research Society, 9, 805-809. 
‘Based on J. Pooley (1994), “Integrated Production and Distribution Facility Planning for Ault 


Foods,” Interfaces, 24:4, 113-121. 
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11-36 Space structures” designed for zero gravity 
have no structural weight to support and nothing 
to which a foundation can be attached. The struc- 
ture needs only to withstand vibrations in space. 
This is accomplished by replacing truss members 
with dampers at a given number p sites among j = 
1,...,n candidate locations throughout the struc- 
ture. Engineering analysis can identify the principal 
strain modesi = 1,...,m, and estimate fractions dj; 
of total modal strain energy imparted in mode i to 
truss site j. The best design places dampers to absorb 
as much energy as possible. Specifically we want to 
maximize the minimum total of d,; reaching chosen 
sites over all modes i. Formulate an ILP model to 
choose an optimal design in terms of p, the dj;, and 
the decision variables (j = 1,..., 7) 

a 1 if a damper is placed at j 

=) 0 — otherwise 


z 4 smallest modal d-total 
(Hint: Maximize z.) 


11-37 The National Cancer Institute!? has received 
proposals from 22 states to participate in its newest 


smoking intervention study. The first] = 1,...,5 are 
from the Northeast region, the next j = 6,...,11 
from the Southeast, numbers j = 12,...,17 from 


the Midwest, and the last 7 = 18,...,22 from the 
West. At least 3 are to be selected from each re- 
gion. Each proposal has been evaluated and rated 
with a merit score 7; based on rankings by a panel 
of experts. Selected project budgets b; (in millions 
of dollars) must total no more than the $15 million 
available for the study, and the number of smokers 
s; (in millions) living in chosen states must sum to 
at least 11 million. Proposals 7 = 2,7, 11,19 come 
from states in the highest quartile with respect to 
the fraction of the population that smokes; propos- 
als j = 1,4, 13, 14, 21 come from the lowest quartile. 
At least 2 states must be selected from each of these 
outlier groups. Formulate an ILP model to choose 
a maximum merit feasible set of proposals to fund 


Conference, part 6, 3086-3095. 


Journal of the Operational Research Society, 46, 919-929. 


Based on R. K. Kincaid and R. T. Berger (1993), “Damper Placement for the CSI-Phase I Evo- 
lutionary Model,” 34th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials 


'3Based on N. G. Hall, J. C. Hershey, L. G. Kessler, and R. C. Spotts (1992), “A Model for Making 
Project Funding Decisions at the National Cancer Institute,” Operations Research, 40, 1040-1052. 
4Based on J. F. Campbell and A. Langevin (1995), “The Snow Disposal Assignment Problem,” 


5Based on G. Rose, D. W. Bennett, and A. T. Evans (1992), “Locating and Sizing Road Maintenance 
Depots,” European Journal of Operational Research, 63, 151-163. 
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using the decision variables (j =1,..., 22) 


1 if proposal 7 is selected | 
Xj & 


0 otherwise | 


11-38 Every year the city of Montreal!* must remove | 
large quantities of snow from sectors i = 1,..., 60 
of the city. Each sector is assigned to one of sites 
j = 1,...,20 as its primary disposal point. From 
prior-year history, planners have been able to esti- 
mate the expected volume of snowfall f; (in cubic 
meters) in each sector and the capacity u, (in cubic 
meters) of each disposal site. They also know the 
travel distance d;; from each sector to each disposal 
site. Removal rates also have to be considered. The 
total of hourly removal rates r; (in m3/hr) associated 
with sectors assigned to any disposal site must not 
exceed its receiving rate b; (in m?/hr). Formulate an 
ILP model to select an assignment that minimizes to- 
tal distance times volume moved using the decision 
variables; =i 7. 6037 = hs 25.20) 


4 il 
xij = 0 


11-39 Highway maintenance in Australia’ is per- 
formed by crews operating out of maintenance de- 
pots. The Victoria region is planning a major realign- 
ments of its depots to provide more effective service 
tothei =1,...,276 highway segments in the region. 
A total of 14 sites must be selected from among the 
possible depot locations j = 1,...,36. Review of 
the possibilities has determined the indicators 


if sector 7 is assigned to disposal j 


otherwise 


1 if a depot at j is close enough 
a; to provide good service to segment i 
a 


0 otherwise 


Not all segments will be able to receive this good ser- 
vice, so planners wish to select 14 locations to min- 
imize to sum of service requirements s; at segments 
with inadequate service. Formulate an ILP model 
to select an optimal collection of depots using the 


decision variables (i = 1,...,276;7 =1,..., 36) 
1 if depot j is selected 


sg 
‘=| 0 otherwise 
1 if segment i goes inadequately 
A covered 


Ji 
0 otherwise 


11-40 Mobil Oil Corporation'® serves 600,000 cus- 
tomers with 430 tanktrucks operating out of 120 bulk 
terminals. As illustrated in the following figure, tank- 
trucks have several compartments c = 1,...,n of 
varying capacity ue. 


compartment cdmpartment compartment {compartment 
al 2Z 3 4 


The final stage of distribution planning is to allocate 
outgoing gasoline products p = 1,...,m to com- 
partments. The various products ordered are placed 
in one or more compartments, but each compart- 
ment can contain only one product. To avoid over- 
filling, the total gallons of any product loaded can 
be reduced from the ordered volume v, by up to 
by gallons. The loading procedure seeks to min- 
imize the sum of these underloadings while meet- 
ing all other requirements. Formulate an ILP model 
of this loading problem using the decision variables 
Ws. Ne — 1... 5 7) 


S| 


Xp,c & gallons of product p loaded in 
compartment c 


1 if product p is loaded in 


pice compartment c 


0 otherwise 
Zp & gallons underloading of product p 


11-41 Each of the 11 nurses!” at Rosey Retirement 
Home works a total of 10 days within each 2-week 
period, alternating between a compatible pair of 


i a a a Raa 


623 


Exercises 


weekly schedules taken from those depicted in the 
following table (1 = work, 0 = off). 


Shift 

Dayo dd 3 ess 
iRl j 2), 0 eo 
Peear de te 
Se es ey Ae 
go Weel ae 
5 bf ft 4 fF 
6 Roar a 7 
are Inga atG ES) 4. 4 


Compatible weekly schedule pairs are C A {(1, 4), 
(1,5), (1,6), 2, 4), (2,5), (2, 6), (3,4), (3,5), 3, 6), 
(4,1), (4,2), (5,2), (5,3), (6, 3)}, and the number 
working each shift should be constant from week to 
week. At least rz nurses must be on duty on days 
d =1,...,7, but excesses are allowed if mandated 
by the shift patterns. Rosey’s objective is to mini- 
mize the sum of any such overstaffing. Formulate an 
ILP model to decide an optimal cyclic schedule for 
the staff of 11 using the decision variables ((i, j) € C, 
Gh vilgemiee7)) 


x;; 4 number of nurses working shift i then shift j 
Zq 2 excess number of nurses scheduled on day d 


11-42 Regional Bell telephone operating compa- 
nies,'* which buy many products from a much smaller 
number of suppliers, often solicit discounts based on 
the total dollar volume of business awarded to partic- 
ular suppliers. For example, suppliers i = 1,...,25 
might be required to quote base prices p;; for needed 
products 7 = 1,...,200, together with upper lim- 
its biz, kK = 1,...,5 for ranges on total dollar vol- 
ume and corresponding discount fractions d,, that 
increase withk. Then the actual cost to the telephone 
company of supplier i’s goods will be (1 — d;,1) times 
the total base price of those goods if that total dol- 
lar value falls within the interval [ujo, u;1], 1 — dj) 
if the total dollar volume falls within [u;1, u;2], and 
so on (assume that uo 4 0, u;5 > any feasible dol- 
lar volume). The company wants to choose bids to 
equal or exceed all required product quantities 7; at 
least total discounted cost. Formulate an ILP model 


l6Based on G. G. Brown, C. J. Ellis, G. W. Graves, and D. Ronen (1987), “Real-Time, Wide Area 


Dispatch of Mobil Tank Trucks,” Interfaces, 17:1, 107-120. 


'7Based on E. S. Rosenbloom and N. FE. Goertzen (1987), “Cyclic Nurse Scheduling,” European 


Journal of Operational Research, 31, 19-23. 


18Based on P. Katz, A. Sadrian, and P. Tendick (1994), “Telephone Companies Analyze Price Quo- 
tations with Bellcore’s PDSS Software,” Interfaces, 24:1, 50-63. 


624 Chapter Eleven e Discrete Optimization Models 


of this volume discount problem using the decision 
variables(i = 1,...,25;j7 =1,...,200;k =1,...,5) 


xij & quantity of product j purchased from 
supplier i 


wiz & dollar volume of goods from supplier i 
when discount range k applies 


1 if discount rang &k applies for 
Vig supplier i 


0 otherwise 


11-43 The small Adele” textile company knits prod- 
ucts p = 1,...,79 on a variety of machines m = 
1,...,48 to meet know output quotas g, pounds 
for the next week. The variable cost per pound 
of making product p on machine m is known ¢,p. 
Machines operate with changeable cylinder types 
j=1,...,14, which have different combinations of 
knitting needles, and thus yield different quantities 
m,j,p (pounds) of product p per hour on machine m. 
A total of 100 hours is available on each machine 
over the next week, but a setup time s,,; must be 
deducted for each cylinder type j used on each ma- 
chine m. Adele wants to find a minimum total vari- 
able cost schedule that conforms to all constraints. 
Formulate an ILP model of this production schedul- 
ing problem using decision variables (m = 1,..., 48; 
fee el p= Ly: 0, 79) 


Xm,jp = pounds of product p made on machine 
m with cylinder type j 


1 if cylinder type j is used on 
Vn machine 


0 otherwise 


11-44 Mail Order Mart (MOM)” will ship g; pounds 
of small-order novelty goods over the next week to 
regions j = 1,...,27 of the United States. MOM’s 
distribution facility is located in New England region 
1. Orders can be direct shipped from the distribu- 
tion center by small parcel carriers at cost p;,; per 
pound. An often cheaper alternative is to drop-ship 
(i.e., group) the week’s orders for a region j into a 
bulk quantity that can be sent by common carrier 
freight to an intermediate point in region i for c; per 


pound (c,; 4 0), and then be direct-shipped from 
on to j at small-parcel cost p;; per pound. Howev= 
common carriers require a minimum of 1000 poun- 
per shipment. MOM wants to identify the least tow: 
cost way to meet this week’s shipping needs. Form: 


late an ILP model of this shipping problem using 


decision variables (i, 7 = 1, ...,27) 
1 if goods bound for region j are drop 
Nag shipped via region i 


0 otherwise 


(x1, = 1 implies direct shipping to j.) 
11-45 Gas turbine engines”! have the following = 
dial assembly of nozzle guides 


located immediately upstream from each rotor. ~ 
purpose of the vanes is to spread flow uniformly 
the rotor, which improves its efficiency mater 
During engine maintenance, the 55 old vanes are » 
moved and replaced by new and refurbished 

i =1,...,55 marked with previously assessed = 
formance measures a; and b; of the two faces. ~ 
effect of each nozzle slot is greatly impacted by 
sum of the a-value for the vane placed on one 
and the b-value for the vane installed on the 
Maintenance personnel want to chosen a (coun 
clockwise) arrangement of vanes around the assom 
bly to balance this performance by minimizing 
total of squared deviations between each resu 
a+bsum and known target value t. Show that 


1Based on U. Akine (1993), “A Practical Approach to Lot and Setup Scheduling at a Textile Firm,” 


ITE Transactions, 25, 54-64. 


20Based on L. S. Franz and J. Woodmansee (1993), “Zone Skipping vs. Direct Shipment of Small Or- 
ders: Integrating Order Processing and Optimization,” Computers and Operations Research, 20, 467-475. 

?1Based on R. D. Plante, T. J. Lowe, and R. Chandrasekaran (1987), “The Product Matrix Traveling 
Salesman Problem: An Application and Solution Heuristic,” Operations Research, 35, 772-783. 


vane arrangement task can be viewed as a traveling 
salesman problem and write an expression for the 
corresponding point 7 to point j costs. 

11-46 A new freight airline” is designing a hub- 
and-spoke system for its operations. From a total 
of 34 airports to be served, 3 will be selected as hubs. 
Then (one-way) airport-to-airport freight quantities 
f,; will be routed via the hubs (f;,; = 0 for all 7). That 
is, flow from i to will begin at 7, go to the unique hub 
k for i, then pass to the (possibly same) hub ¢ for j 
before being shipped on to j. The goal is to minimize 
the total of flows times unit transportation costs cj; 
taking into account a 30% savings for flows between 
hubs that results from economies of scale (c;; = 0 
for all 7). 


(a) Explain why appropriate decision variables for 
an integer programming model of this hub de- 
sign problem are (i,k = 1,...,34) 


it if airport i is assigned to a hub 


Gs ee atk 
0 otherwise 
rn ie! if a hub opens at k 
Mg AO reer 
SUGGESTED READING 


Nemhauser, George L., and Laurence A. Wolsey 
(1988), Integer and Combinatorial Optimization, 
Wiley, New York. 
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Exercises 


(b) Use only the x;, (i.e., disregard whether hubs 
are open) to formulate a quadratic objective 
function summing origin-to-hub, hub-to-hub, 
and hub-to-destination transportation costs for 
each pair of nodes. 

(c) Complete an INLP model of the problem by 
adding linear main constraints and appropriate 
variable-type constraints. 


11-47 In an area with many suburban communi- 
ties, telephone listings are usually grouped into sev- 
eral different books.** Patrons in each community 
1,...,m are covered in exactly one directory 
k =1,...,m. Numbers of patrons p; are known for 
each community, as well as (one-way) calling traf- 
fic levels 1;,; between communities. Engineers seek 
to design books that maximize the traffic among pa- 
trons sharing a common telephone book without list- 
ing more than capacity g patrons in any single direc- 
tory. Formulate an INLP model to select an optimal 
collection of telephone books in terms of the deci- 
sion variables (i=1...., Tk — A ay m) 


e 1 if community i goes in book k 
Xik = ‘ 
0 otherwise 


Parker, R. Gary, and Ronald L. Rardin (1988), 
Discrete Optimization, Academic Press, San Diego, 
Calif. 


22Based on M. E. O'Kelly (1987), “A Quadratic Integer Program for the Location of Interacting Hub 
Facilities,” European Journal of Operational Research, 32, 393-404. 
23Based on S. Chen and C. J. McCallum (1977), “The Application of Management Science to the 


Design of Telephone Directories,” Interfaces, 8:1, 58-69. 
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Discrete 
Optimization Methods 


In Chapter 11 we illustrated the wide range of integer and combinatorial optimization 
models encountered in operations research practice. Some are linear programs with 
a few discrete side constraints; others are still linear but involve only combinatorial 
decision variables; still others are both nonlinear and combinatorial. Every one 
includes logical decisions that just cannot be modeled validly as continuous, so most 
lack the elegant tractability of the LP and network models studied in earlier chapters. 

Diminished tractability does not imply dimished importance. Discrete opti- 
mization models such as those presented in Chapter 11 all represent critical decision 
problems in engineering and management that must somehow be confronted. Even 
partial analysis can prove enormously valuable. 

It should not surprise that discrete optimization methods span a range as wide 
as the models they address. In contrast to, say, linear programming, where a few 
prominent algorithms have proved adequate for the overwhelming majority of mod- 
els, success in discrete optimization often requires methods cleverly specialized to an 
individual application. Still, there are common themes. In this chapter we introduce 
the best known. 


12.1 SOLVING BY TOTAL ENUMERATION 


Beginning students often find counterintuitive the idea that discrete optimization 
problems are more difficult than their continuous analogs. The algebra of LP algo- 
rithms in Chapters 5 and 6is rather daunting. By comparison, a discrete model, which 
has only a finite number of choices for decision variables, can seem refreshingly easy. 
Why not just try them all and keep the best feasible solution as optimal? 

Although naive, this point of view contains a kernel of wisdom. 


ables, the most effective 
direct: enumeration of all the possibilities. 
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Total Enumeration 


To be more specific, total or complete enumeration requires checking all possibilities 
implied by discrete variable values. 


2 | Total enumeration solves a discrete optimization by trying all possible 


sponding choice of any continuous variables. Among combinations yielding a 
feasible solution, those with the best objective function value are optimal. 
Swedish Steel All-or-Nothing Example 
We can illustrate with the discrete version of our Swedish Steel example formulated 
in model (11.2) (Section 11.1): 
min 16(75)y; + 10(250)y2 + 8x3 + 9x4 + 48x5 + 60x6 + 53x7 
st. 75y1 + 250y2 +43 +4 +%5 +%6 +27 = 1000 
0.0080(75)y1 + 0.0070(250)y2 + 0.0085x3 + 0.0040x4 = 0.0065(1000) 
0.0080(75)y1 + 0.0070(250)y2 + 0.0085x3 + 0.0040x4 < 0.0075(1000) 


0.180(75)y1 + 0.032(250)y2 + 1.0x5 > 0.030(1000) 

0.180(75)y1 + 0.032(250)y2 + 1.0x5 < 0.035(1000) (12.1 
0.120(75)y1 + 0.011(250)y2 + 1.0x6 > 0.010(1000) ’ 
0.120(75)y; + 0.011(250)y> + 1.0x6 < 0.012(1000) 

0.001(250)y2 + 1.0x7 > 0.011(1000) | 
0.001 (250) y2 + 1.0x7 < 0.013(1000) | 
2 eleecuerery 8, C7) > 0 

vM,y¥2 = Oorl 


In this version the first two sources of scrap iron have to be entered on an all-or 
nothing basis modeled with discrete variables. The other five sources can be em- 
ployed in any nonnegative amount. 

There are 2 possible values for y; and 2 for y2, ora total of2-2 =4 combinations 
to enumerate. Table 12.1 provides details. Third option y; = 1, yz = 0 yields the 
optimal solution with objective value 9540.3. 


TABLE 12.1 Enumeration of the Swedish Steel All-or-Nothing Model 


Discrete Corresponding Continuous Objective 
Combination Solution Value 
y,=0, y2=0 x3=814.3, x,=1146, x5=30.0, x,=10.0, x,=1.1 9914.1 
y,=0, yo=l x,;=637.9, x,= 82.0, x5=22.0, xp= 7.3, x,=0.9 9877.3 
Yi=1, yy=0  %3= 727.6, xX4=178.8, x5;=16.5, xg= 1.0, x,=11 9540.3 
Yy=1, yy=l  %,=552.8, x,=112.9, x,= 8.5, x,= 0.0, x,=0.9 9591.1 


Since this model has both discrete and continuous variables, each case enu- 
merated requires solving a continuous optimization over variables x3,..., x7 to find 
the best continuous values to go with the choice of discrete variables selected. For 
example, fixing yj = y2 = 0 in model (12.1) leaves the linear program 


12.1 Solving by Total Enumeration 


min 16(75)(0) + 10(250)(0) + 8x3 + 9x4 + 48xs 4 60x6 + 53x7 


st.  75(0) + 250(0) +.x3 + x4 +%5 +246 +47 


= 1000 


0.0080(75) (0) + 0.0070(250) (0) + 0.0085x3 + 0.0040x4 > 0.0065 (1000) 
0.0080(75) (0) + 0.0070(250) (0) + 0.0085x3 + 0.0040x4 < 0.0075 (1000) 


0.180(75)(0) + 0.032(250)(0) + 1.0x5 
0.180(75) (0) + 0.032(250)(0) + 1.0x5 
0.120(75) (0) + 0.011(250) (0) + 1.0x¢ 
0.120(75) (0) + 0.011(250) (0) + 1.0x¢ 
0.001 (250) (0) + 1.0.x, 
0.001(250) (0) + 1.0x7 


> 0.030(1000) 
< 0.035(1000) 
=> 0.010(1000) 
< 0.012(1000) 
> 0.011(1000) 
< 0.013(1000) 


Kanraie aX 7e 


Optimal solution x3 = 814.3, xy = 144.6, x5 = 30.0, x6 = 10.0, x7 = 1.1, completes 
the first case in Table 12.1. 


| SAMPLE EXERCISE 12.1: SOLVING BY TOTAL ENUMERATION 
Solve the following discrete optimization model by total enumeration ; 


max 7x; + 4x2 + 19x3 
s.t. xy +%3<1 
x2 +x3 <1 
X1,%2,x3 =Oorl 


Analysis: Checking the 2? = 8 combinations produces the following table: 


Case 


Objective Objective 


x = (0, 0, 0) 0 x = (1,0, 0) ih 
x = (0, 0, 1) 19 x=(1,0,1) Infeasible 
x =(0, 1,0) 4 x = (1, 1,0) tl 


x=(0,1,1) Infeasible | x=(1,1,1)  Infeasible 


Solution x = (0, 0, 1) is the feasible one with best objective value 19, so it is optimal. 


Exponential Growth of Cases to Enumerate 
Our Swedish Steel example has two discrete decision variables, each with two possible 
values 0 and 1. A total of 

ey = 7 4 


combinations result. 
Similar thinking shows that a model with k binary decision variables would 
have 
DD AIS ON 
—{—>—-——— 
k times 


cases to enumerate. This is exponential growth, with every additional 0-1 variable 
doubling the number of combinations. 
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An analyst can easily run 2? = 4 or 24 = 16 cases—perhaps even 2!° = 1024 
with the aid of a computer. But 2! ~ 10°°, and we know that a discrete model with 
k = 100 binary variables is not particularly large. 

The fastest current computers perform a few billion (10) arithmetic operations 
in asecond. Future computers might well solve the entire linear program associated 
with each choice of discrete variables in that same time. Assume even more, that 
a trillion (10'7) cases could be checked in a single second. Enumeration of a 100- 
variable model would still require 


2100 


1012 
too long for the most patient of decision makers to wait. 


~ 1.27 x 10'8 seconds 402 million centuries 


123 Exponential growth makes tota enumeration impractical with models 
having more than a handful of discrete decision variables. 


SAMPLE EXERCISE 12 2: UNDERSTANDING EXPONENTIAL GRowrH 


Suppose that your personal computer can enumerate one combination of discrete 
values each second of a given mixed-integer program, including solving the implied 
optimization for corresponding continuous variable values. Determine how long it 
would take to totally enumerate instances with 10, 20, 30, and 40 binary variables. 


Analysis: For 10 variables, enumeration would require 
2'9 — 1024 seconds ~ 17.1 minutes 


Each increment of 10 binary variables multiplies the number of combinations by 
2'° = 1024. Thus a case with 20 variables would require 


1024 - 17.1 ~ 17, 500 minutes ~ 12.1 days 


An instance with 30 binary variables would need about 1024 . 12.1 = 12,390 days, 
and one with 40 variables would require nearly 12.7 million days, more than 347 
centuries. 


Nonlinearities 


The practicality of enumeration in mixed cases is also limited by the continuous 
problem that remains when discrete variables are enumerated. Our Swedish Steel 
case was an integer linear program (ILP), so that each case involved only solving an 
LP. If the remaining continuous model had been nonlinear, even evaluating the cases 
could have been a difficult task. 


nes RELAXATIONS OF DISCRETE OPTIMIZATION MODELS 
AND THEIR USES 


Because analysis of discrete optimization models is usually hard, it is natural tc 
look for related but easier formulations that can aid in the analysis. Relaxations are 


". ‘ __ nn 
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auxiliary optimization models of this sort formed by weakening either the constraints 
or the objective function of the given discrete model. 


EXAMPLE 12.1: BISON BOOSTERS 


Before considering relaxation in the more realistic circumstances of models in Chap- 
ter 11, it will help to develop a more compact (albeit highly artificial) example. Con- 
sider the dilemma of the Bison Boosters club supporting the local atheletic team. 

The Boosters are trying to decide what fundraising projects to undertake at the 
next country fair. One option is customized T-shirts, which will sell for $20 each; the 
other is sweatshirts selling for $30. History shows that everything offered for sale 
will be sold before the fair is over. 

Materials to make the shirts are all donated by local merchants, but the Boosters 
must rent the equipment for customization. Different processes are involved, with 
the T-shirt equipment renting at $550 for the period up to the fair, and the sweatshirt 
equipment for $720. Display space presents another consideration. The Boosters 
have only 300 square feet of display wall area at the fair, and T-shirts will consume 
1.5 square feet each, sweatshirts 4 square feet each. What plan will net the most 
income? 

Certainly this problem centers on making shirts, so decision variables will in- 
clude 


x, 4 number of T-shirts made and sold 
x2 4 number of sweatshirts made and sold 


However, the Boosters also confront discrete decisions on whether to rent equip- 
ment: 


y, 41 if T-shirt equipment is rented and = 0 otherwise 
yy 4.1 if sweatshirt equipment is rented and = 0 otherwise 


Using these decision variables, the Boosters’ dilemma can be modeled: 


max 20x; + 30x. —550y; — 720y2 (net income) 


st. 1.5x1 + 4x2 < 300 (display space) 
x1 < 2001 (T-shirts if equipment) 
x2 < 75y2 (sweatshirts if equipment) G22) 
X1,X2 = 0 
y1,¥2 =0orl 


The objective function maximizes net income, and the first main constraint enforces 
the display space limit. The next two constraints provide the switching we have seen 
in other models. Any sufficiently large big-M could be used as the y; coefficient 
in these constraints. Values in (12.2) derive from the greatest production possible 
sithin the 300 square feet display limit. Coefficients 300/1.5 = 200 for T-shirts and 
300/4 = 75 for sweatshirts introduce no limitation if y’s equal 1, yet switch off all 
production if y’s equal 0. 

Enumeration of the 4 combinations of y; and y2 values easily establishes that 
the Boosters should make only T-shirts. The unique optimal solution is x} = 200, 
<= =0, yj = 1, y} = 0, with net income $3450. 
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Constraint Relaxations 
Relaxations may weaken either the objective function or the constraints, but the 


elementary ones we explore in this book nearly all focus on constraints. A constraint 
relaxation produces an easier model by dropping or easing some constraints. 


12.4 Model (P) is a constraint relaxation of model (P) if every feasible solution. 
to (P) is also feasible in (P) and both models have the same objective function. 


New feasible solutions may be allowed, but none should be lost. 

Table 12.2 shows several constraint relaxations of the tiny Bison Boosters model 
(12.2). The first simply doubles capacities. The result is certainly a relaxation, because 
every solution fitting within the true capacity of 300 square feet will also fit within 
twice as much area. Still, this relaxation gains us little. 


TABLE 12.2 Constraint Relaxations of Bison 
Boosters Model 


Revised Constraints Discussion 
1.5x, + 4x, < 600 Doubled capacities. Relaxation 
x, <400y, optimum: £, = 400, ¥, = 0, 9, = 1, 
Xy S$ 150y, J, = 0, net income $7450 
X1X22 0 
V4, y,=Oorl 
xX, S$ 200y, Dropped first constraint. Relaxation 
XyS75y5 optimum: ¥, = 200, %, = 75, f, =1, 
x4,X2 20 J, = 1, net income $4980 
V1,¥2=0orl 
1.5x, + 4x, $ 300 Linear programming relaxation with 
x, <200y, discrete variables treated as continu- 
XS 75yy ous. Relaxation optimum: ¥; = 200, 
X41, X, 20 X,= 0, 9, = 1, §, = 0, net income 
0S y,<1 $3450 
OS yy S1 


12.5 | Relaxations should be significantly more tractable than the models the 
relax, so that deeper analysis 1 is practical. : 


Doubling capacities fails this requirement because the character of the model is 
unchanged. 

The second relaxation of Table 12.2 is more on track. Dropping the first con- 
straint delinks decisions about the two types of shirts. It then becomes much easier to 
compute a (relaxation) optimal solution. We need only decide one by one whether 
the maximum production now allowed each x; when its y; = 1 justifies the fixed cost 
of equipment rental. Both do. 


SAMPLE EXERCISE 12.3: RECOGNIZING RELAXATIONS 


Determine whether or not each of the following mixed- sinvewed programs is a con- 
straint relaxation of 
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min 3x, + 6x2 + 7x3 +.%4 

st. 2x, +X2 +%3+10x4 => 100 
Xy tx. +43 <1 
X1,X2,x%3 =Oorl 


x4 >0 
(a) min 3x; + 6x. +7x34+%4 (b) min 3x, + 6x2 + 7x3 +4 
st. 2x; +x. +%3 + 10x4 > 100 st. 2x, + x2 +3 +10x4 > 200 
X1,%2,x3 =Oorl xy try +4%3<1 
x4 > 0 X1,X2.,x3 =Oorl 
x4 > 0 
(c) min 3x; + 6x2 + 7x3 + x4 (4) min 3x, + 6%) +7x3 +24 
st. 2x, +X. +x3+10x4 > 100 St. 2x, +X2 +x3 +10x4 > 100 
xy tx. +43 <1 Xp +X. +4x3<1 
X14 X2,%3,%4 = 0 1>%x,>0,1>m>0, 


Ea On = 0 


Analysis: We apply definition ; 


(a) This model is a constraint relaxation because it is formed by dropping the second 
main constraint. Certainly, every solution feasible in the original model remains so 
with fewer constraints. 


(b) This model is not a relaxation. The only change, which is increasing the right- 
hand side by 100, to 200, eliminates previously feasible solutions. One example is 
x = (0,0, 0, 10). 

(c) This model is a relaxation. Allowing x, x2, and x3 to take on any nonnegative 
value—rather than just 0 or 1—cannot eliminate previously feasible solutions. 


(d) This model is also a relaxation. Allowing x1, x2, and x3 to take on any values in 
the interval [0, 1] precludes none of their truly feasible values. 


Linear Programming Relaxations 


The third case in Table 12.2 illustrates the best known and most used of all constraint 
relaxation forms: linear programming, or more generally, continuous relaxations. 
(linear programming relaxations if the given 
reatine any discrete vari- 


12.6 | Continuous relaxation 
model is an integer linear p 


In the real Bison Boosters model, each y; must equal 0 or 1. In the continuous 
relaxation we also admit fractions, replacing each 


Certainly, no feasible solutions are lost by allowing both fractional and integer 
choices for discrete variables, so the process does produce a valid relaxation. More 
important, the relaxed model usually proves significantly more tractable. 
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Our Bison Boosters model is an integer linear program (ILP), linear in all 
aspects except the discreteness of y; and y2. Thus, relaxing discrete variables to 
continuous leaves a linear program to solve—the linear programming relaxation; we 
have already expended several chapters of this book showing how effectively linear 
programs can be analyzed. 


12.7 | LP relaxations of integer linear programs are by far the most used relax- 
ation forms because they bring all the power of linear programming to bear on 
analysis of the given is ete mod = . 


aca the iiear'p programming relaxation of the following re eet program: 
min 15x; + 2x2 — 4x3 + 10x4 
st. x3—XxX4 <0 
Xy + 2x2 + 4x3 + 8x4 = 20 
x2 +x4<1 
x, >0 
X2,X3,X4 = Oor 1 


Analysis: Following definition 12.6], we replace 0-1 constraints x; = Oor 1 by 
€ [0, 1] to obtain the LP relaxation 


min 15x, + 2x2 — 4x3 + 10x4 
st. x3 —X4 <0 
xy + 2x2 + 4x3 + 8x4 = 20 
x2 +x4 <1 
x, >0 
LS) SU, 1 Six5 > 0jokeaged 


Proving Infeasibility with Relaxations 


Exactly what do relaxations add to our analysis of discrete optimization models? 
One thing is prove infeasibility. 

Suppose that a constraint relaxation comes out infeasible. Then it has no so- 
lutions at all. Since every solution to the full model must also be feasible in the 
relaxation, it follows that the original model was also infeasible. By analyzing the 
relaxation we have learned a critical fact about the model of real interest. 


Use neat programming relaxation to eetabiheh that the following discrete optim 
tion model is infeasible: 
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min 8x; +2x> 
st. X41 —-X2 > 2 
=x, +x. >-—1 
X1,X2 > 0 and integer 
Analysis: The linear programming relaxation of this model is 
min 8x; + 2x2 
St. x1 —-X%.>2 
i tg SL 
X1,%2 > 0 
It is clearly infeasible, because the two main constraints can be written 
if oe Dh 
X1—-X. <1 


Thus by principle , the given integer program is also infeasible. Any solutions 
satisfying all constraints would also have to be feasible in the relaxation. 


Solution Value Bounds from Relaxations 


Figure 12.1 illustrates how relaxations also give us bounds on optimal solution values. 
Constraint relaxations expand the feasible set, allowing more candidates for relax- 
ation optimum. The relaxation optimal value, which is the best over the expanded 
set of solutions, must then equal or improve on the best feasible solution value to the 
true model. 


feasible solutions in relaxation 


feasible solutions in thue model 


FIGURE 12.1 Relaxations and Optimality 


All three relaxations in Table 12.2 illustrate the maximize case. The optima! 
solution value of the Bison Boosters model (12.2) is $3450. One of the cases in Table 
12.2 yields exactly this value. The others produce higher estimates of net incom: 


All provide the upper bound guaranteed in principle 
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With the Bison Boosters model, which is so small that it is easily solved opti- 
mally, relaxation bounds offer little new insight. A better sense of their value comes 
from considering the somewhat larger EMS model (11.8) of Section 11.3 that mini- 
mizes the number of stations to cover the 20 metropolitan districts: 


10 
min y Xj 
j=l 


(number of sites) 


Sit. 3% > 1 (district 1) 
x1 +X > 1 (district 2) 
x1 +.x3 > 1 (district 3) 
x3 > 1 (district 4) 
x3 > 1 (district 5) 
x2 > 1 (district 6) 
Xo +X4 > 1 (district 7) 
x3 4+%4 > 1 (district 8) 
Xg > 1 (district 9) 
x4 +X6 > 1 (district 10) (12.3) 
X4 + X5 > 1 (district 11) 
X4+x5 + X6 > 1 (district 12) 
X4 + x5 +47 > 1 (district 13) 
Xg + X9 > 1 (district 14) 
x6 +X9 > 1 (district 15) 
Xs + X6 > 1 (district 16) 
xX5+x7+X10 > 1. (district 17) 
Xg + X9 > 1 (district 18) 
X9 + X10 > 1 (district 19) 
X10 > 1 (district 20) 
Kiss o-oo = Viorel 


How many stations does this model imply? Even with just 10 discrete variables. 
the answer is hardly obvious. But if we replace each x; = 0 or 1 constraint by 0 < 
x; < 1, the resulting linear programming relaxation can be solved quickly with say. 
the simplex algorithm. An optimal solution is 

Xi ee) 
= X3 = Xe = tay. — Ll (12.4) 
Hie Ks =e a a 


with optimal value 6.0. Without looking any further into the discrete model, we can 
conclude that at least 6 EMS sites will be required because this LP relaxation value 


provides a lower bound (principle : 


each of the following integer programs. 
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(a) max x, +%2+%3 
s.t. Xp +X. <1 
NG SetsessS 1 
x, +x3<1 
x1,%2,%3 =Oorl 


(b) min 20x41 + 9x2 + 7x3 
Sate 10x, + 4x. + 3x3 >7 
X1,%X%2,%3 = Oorl 


Analysis: 


(a) Clearly, only one of the variables in this model can = 1, so the optimal solution 
value is 1. Corresponding linear programming relaxation 


max XxX, +%2+%3 
Sit) = q 4s A 
xy tx3 <1 
x2 +x3 <1 
1 > x4, X2, «3 >0 


yields optimal solution x = G. 7 5) with objective value 3, In accord with principle 


112.9 |, relaxation value 3 is an upper bound on the true optimal value 1 of this 
maximize model. 


(b) Total enumeration shows that an optimal solution to this minimizing ILP is x = 
(0, 1, 1) with value 16. Its linear programming relaxation is 


min 20x, + 9x2 + 7x3 
st. 10x, + 4x2 +3x3 > 7 
Lin x3 0 


with optimal solution x = (Z. 0, 0) and value 14. Demonstrating principle [12.9 |, 
relaxation value 14 provides a lower bound on true optimal value 16. 


Optimal Solutions from Relaxations 


Sometimes relaxations not only bound the optimal value of the corresponding dis- 
crete model but produce an optimal solution. 


Another look at Figure 12.1 will show why. All (shaded-area) feasible solutions 
to the original discrete model must also belong to the larger relaxation feasible set. 
If the relaxation optimum happens to be one of them, it has as good an objective 
function value as any feasible solution to the relaxation. In particular, it has as good 
an objective function value as any feasible solution in the original model. It must be 
optimal in the full model. 

The third, linear programming relaxation of the Bison Boosters model in Table 
12.2 illustrates. Even though integrality was not required of y-components in the 


637 


638 Chapter Twelve e Discrete Optimization Methods 


relaxation optimal solution 
%,=200, x2,=0, W=1, w=0 


it happened anyway. This relaxation optimum is feasible in the full discrete model 
and so optimal there. 


SAMPLE EXERCISE 12.7: OBTAINING OPTIMAL SOLUTIONS FROM RELAXATIONS _ 


Compute (by inspection) optimal solutions to each of the following relaxations, and 
determine whether we can conclude that the relaxation optimum is optimal in the 
original model. 


(a) The linear programming relaxation of 


max 20x; + 8x2 + 2x3 
s.t. xy tx. +43 <1 
X1,%2,x%3 =Oorl 


(b) The linear programming relaxation of 


max xX, +%2+4%3 

S.t. x1 +x <1 
xy +%3<1 
x2 +x3 <1 
X1,X2,x3 =Oorl 


(c) The relaxation obtained by dropping the first main constraint of 
min 2x; + 4x2 + 8x3 
St. xy +X. +43 <2 
10x; + 3x. + 4x3 > 8 
X1,%2,%3 = Oorl 
Analysis: 
(a) The linear programming relaxation of this model is 


max 20x; + 8x2 + 2x3 
Sit Xj $x. +%3<1 
1 > x4, x2, x3 = 0 
with obvious optimal solution x = (1,0, 0). Since this solution is also feasible in the 
original model, it follows from principle | 12.10 | that it is optimal there. 
(b) The linear programming relaxation of this model is 
max xX +X%2+%3 
s.t. xy tx <1 
xX. +x3<1 
xy +x3 <1 
D2; age 0 
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with optimal solution x = G, 5, 3): Since this solution violates integrality require- 
ments in the original model, it is infeasible there. It could not be optimal. 


(c) The indicated relaxation is 
min 2x; + 4x2 + 8x3 
st. 10x, + 3x. +x3 >8 
X1,%2,%3 =Oorl 


with obvious optimal solution x = (1, 0, 0). This relaxation optimum satisfies relaxed 
constraint 


X1 +xX2+%3 <2 


and so is feasible in the original model. It follows from principle | 12.10 that it is 
optimal in the full integer program. 


Rounded Solutions from Relaxations 


When principle | 12.10 | applies, relaxation completely solves a hard discrete opti- 
mization model. More commonly, things are not that simple. As with the EMS 
solution (12.4) above, relaxation optima usually violate some constraints of the true 
model. 

All is hardly lost. First, we have the bound of principle . We may also 
have a starting point for constructing a good heuristic solution to the full discrete 
model. 


13411 Many relaxations produce optimal solutions that are easily “rounded” to 


Consider, for example, the EMS solution (12.4). The nature of model con- 
straints (12.3), > form with nonnegative coefficients on the left-hand side, means 
that feasibility of a solution is not lost if we increase some of its components. Begin- 
ning from the LP relaxation optimum and rounding up produces the approximate 
optimal solution 


*, =[%] = [0] =0 
LS ae 


43 == (es) = 1! =1 


& = [i] =] =1 

Xs = [x7] =f] =1 

eee =| (12.5) 
X6 = [Xe] =[3z] =1 


x7 = [x7] = [0] =0 
Xs = [xg] =[1] =1 
[%o9] =[31=1 
X10 = [x10] = [1] =1 


& 
oO 
I 
® 
oO 
—_ 
I 
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with value ey 4; = 8. Here ceiling notation 

[x] A least integer greater than or equal to x 
The corresponding floor notation 

|x| 4 greatest integer less that or equal to x 


Heuristic optimum x may not be truly optimal, but it does satisfy all constraints. 
Where time permits no deeper analysis, this rounded relaxation solution might well 
suffice. Also, feasible solutions provide bounds to complement those obtained from 


the optimal relaxation solution value (principle ys 


optimal value, and any cae feasible solution to a minimizing discrete oe 
timization problem provides an upper bound. 


Set covering relaxation optima like (12.4) are particularly easy to round, be- 
cause of the unusually simple form of the constraints. Many other forms admit 
similar rounding. Some round infeasible relaxation solutions up, some round down, 
and some do other straightforward patching. Details vary with model form. 

Unfortunately, there are some discrete models that just do not round. For an 
example, return to our AA airline crew scheduling model (11.10) (Section 11.3). 
Its set partitioning form closely resembles the set covering case we just rounded 
easily. But set partitioning involves equality constraints. Each time we round some 
infeasible x; up to 1 or down to 0, other variables sharing constraints with that x; 
will also have to be adjusted if feasibility is to be preserved. Much more complex 
rounding schemes are required, and success cannot be guaranteed. 


SAMPLE EXERCISE 12.8: ROUNDING RELAXATION OPTIMA 


In each of the following integer linear programs, develop and apply a scheme for 
rounding the indicated LP relaxation optimum to an approximate solution for the 
full model. Also, indicate the best lower and upper bounds on the optimal integer 
solution value available from relaxation and rounding. 


(a) min 10x; + 8x.+18x3 with LP relaxation optimum x = (0, 1, ;) 
st. 2x, + 4x2 +713 >5 
xy +22 +%3>1 
X1,X2,x3 =Oorl 
(b) max 40x; +2x.+18x3 with LP relaxation optimum & = (1, 0, 3) 
s.t. 2x1 + 11x2 + 7x3 <5 
Xy +x2+x3 <2 


X1,X2,x3 =Oorl 


ue i 


| 
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(c) min 3x, + 5x2 + 20x3 + 14x4 with LP relaxation optimum x = es q, x, a) 
Si Ae x sll 

3x1 + 6x2 = 50 

x1 < 11x 

Kes 11x4 

X1,%X%2 > 0 


X30 = Ororal 


Analysis: 


(a) All main constraints of this model are > form, and coefficients on the left-hand 
side are nonnegative. Thus increasing feasible variable values cannot cause a viola- 
tion. We may round up to integer-feasible solution 
[X] = ([0], 111,151) = 0,1, 1) 
Substituting this solution in the objective function gives an upper bound (princi- 
ple | 12.12) of 26 on the optimal value. The corresponding lower bound, which is 
obtained by substituting the relaxation optimal solution (principle ), is 10.57. 
(b) All main constraints of this model are < form, and coefficients on the left-hand 
side are nonnegative. Thus decreasing feasible variable values cannot cause a viola- 
tion. We may round down to integer-feasible solution 
Lé = (L1J, LO}, 13.) = 2,0, 0) 
Substituting this solution in the objective function gives a lower bound (principle 
12.12 l) of 40 on the optimal value. The corresponding upper bound, which is ob- 
tained by substituting the relaxation optimal solution (principle ), is 47.71. 
c) Each of the discrete variables in this mixed-integer linear program occurs in 


ynly one < constraint on the right-hand side. Thus increasing x3 and x4 from their 
relaxation values cannot lose feasiblity. We may round up to 


16 17 164 7174) _ (16 17 

(3 3°33], 331) = (@. 3, 1,1) 

Notice that continuous variable values were not changed. 
Substituting this solution in the objective function gives an upper bound (prin- 


aple| 12.12 |) of 78.33 on the optimal value. The corresponding lower bound, which 
= obtained by substituting the relaxation optimal solution (principle ), is 61.24. 


12,3 STRONGER LP RELAXATIONS, VALID INEQUALITIES, 
AND LAGRANGIAN RELAXATIONS 


solution (principle | 12.10]), and find rounding much easier (principle | 12.11}) if 


“Se relaxations we employ closely approximate the full model of interest. Strong 
relaxations do just that. 
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Analysts dealing with hard discrete models via relaxations will almost always find it 
worthwhile to look for means to strengthen the relaxations without losing too much 
tractability. 


Stronger LP Relaxations 


To begin, Ict us will focus on the standard linear programming relaxations of ILP 
models. How can we make such LP relaxations strong? The key insight sometimes 
surprises: 


SAMPLE EXERCISE 12.9: UNDERSTANDING STRONGER LP RELAXATION 


Show (by inspection) that even though the two following integer linear program- 
ming models have the same feasible solutions, the second yields a stronger linear 
programming relaxation. 


max XxX, +%2+4%3 max 1 +%2+%3 

SH a Ges a Sa Sst. xy 4+%2 <1 
x1 +%3<1 x, +%x3 <1 
x2 +x3 <1 xg +x3 <1 
X1,X2,x3 =Oorl xy +x. +43 <1 


x1, 42,x3 =Oorl 


Analysis: Both ILPs have the same feasible solutions, 


x = (1,0, 0) 
x® = (0,1,0) 
x9 = (0,0, 1) 


Thus they are both valid models of the same problem. Still, the first has LP relaxatior 
optimum x = G. 5s 5)> and the second has eee! optimum x = (1, 0, 0) (amons 
others). The corresponding relaxation bounds are + 3 and 1, making the second relas- 
ation stronger. In this simple case, in fact, it yields A discrete optimum (via principle 


[12.10]). 


Choosing Big-M Constants 


The “sufficiently large” big-M constants needed in so many models offer one eas 
family where details of ILP modeling affect the LP relaxation. Return, for instance 


— 


a 
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to the tiny Bison Boosters model of (12.2) and Table 12.2. In formulating switching 
constraints x; < 400y; and x2 < 75y2, we constructed values 400 and 75 with a back- 
of-envelope computation. Any sufficiently large M would yield a correct integer 
linear programming model. 

Suppose that we had used 10,000 for both. The new model is 


max 20x; + 30x2 — 550y1 — 720y2 (net income) 


st. 1.5%, + 4x. < 300 (display space) 
x1 < 10, 000y; (T-shirts if equipment) 
eee : (12.6) 
x2 < 10, 000y2 (sweatshirts if equipment) 
%1,x2 =0 
yt, y2 =O0orl 


Recall that the original model (12.2) had relaxation optimum 
4, =200, %=0, fr=1, jo =0 


matching perfectly the discrete optimal solution with value $3450. Its LP relaxation 
was indeed strong. 

Revision (12.6) is every bit as correct as the original (12.2) in the sense that it 
has exactly the same (discrete) feasible set. However, the LP relaxation of (12.6) 
yields optimum 


%=200, %=0, F002, 5,=0 (12.7) 


with value $3989. The value bound $3989 now differs significantly from the true 
optimal value $3450, Also, the relaxation optimal solution has component y, at a 
tiny fractional value. With only (12.7) at hand, it would be hard to tell whether to 
rent or not the T-shirt equipment. 

This contrast between LP relaxations of integer-equivalent models (12.2) and 
(12.6) highlights an important and easy-to-implement principle for strengthening 
relaxations: 


constants. a : 


SAMPLE EXERCISE 12.10: CHOOSING SMALLEST BIG-/’s 


We wish to decide which combination of 2 pharmaceutical facilities should be used 
to produce 80 units of a needed product. One costs $5000 to setup and has variable 
cost $20 unit. The other cost $7000 to setup and has variable cost $15. Both have 
capacity of 200 units. 


(a) Formulate a mixed-integer linear programming model using capacities for needed 
big-M’s. 


5) Strengthen the linear programming relaxation of your model in part (a) by re- 
ducing big-M’s to their smallest valid value. 
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Modeling: 


(a) Using decision variables x; and x2 for the amount produced in each facility, and 
switching variables x3 and x4 to track setups, a valid formulation is 
min 20x; + 15x2 + 5000x3 + 7000x4 
st. xX, +X. = 80 
x, < 200x3 
X2 < 200x4 
Klin A 0 
x3, = Oior t 


Full capacity is available whenever setup cost is paid. 


(b) Although capacities are 200, the problem calls for only 80 units to be produced. 
Thus neither x; nor x2 will ever exceed 80 in an optimal solution. We may strengthen 
the model by reducing big-M constants from 200 to 80, to produce 
min 20x; + 15x2 + 5000x3 + 7000x4 
st. xy +X. = 80 
ies 80x3 
X2 < 80Xx4 
X1,%2 = 0 
x3,xX4 =Oorl 
The reader can verify that this new formulation has relaxation optimum x = (80, 0, 1.| 
with value $6600, versus the original model’s x = (80, 0, .4, 0) at value $3600. 


Valid Inequalities 


Sharpening big-M coefficients is only one of many ways to strengthen LP relaxations. 
We can also add new valid inequality constraints. 


12.16 | A linear inequality is a valid inequality fora given discrete optimization 
model if it holds for all (integer) feasible solutions to the model. 


Relaxations can often be strenghtened dramatically by including valid inequalities 
that are not needed for a correct discrete model. 

Not every valid inequality strengthens a relaxation. For example, all inequality 
constraints of the original formulation are trivially valid because they are satisfied 
by every feasible solution. 


This need to cut off noninteger relaxation solutions is why valid inequalities are 
sometimes called cutting planes. 
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The Tmark facilities location model of Section 11.6 illustrates a classic case. 
The model formulated there is: 


8 14 8 
min ys (yr) x5 + oa Siyi (total fixed cost) 

i=1 j=l i=1 

‘ J 
s.t. yes Bhpiaas I for all j=1,...,14 (carry j load) 

— 14 

1500y;< }" dai; for alli=1,...,8 (minimum at i) (12.8) 

j=l 


14 
Yo dij < 5000y; — foralli=1,...,8 (maximum at i) 


j=! 
x;,; = 0 fomall= ils tay dese 
y, =Oor 1 for alli =1,...,8 


where x; ;if the fraction of region /’s call traffic handled by center 7, y; decides whether 
or not center i is opened, d; is the anticipated call demand from region j, r; jis the 


unit cost of calls from region j to center i, and f; is the fixed cost of opening center i. 


Focus on the third, maximum capacity set of constraints. Each forces discrete 
variable y; to take on a value in the relaxation satisfying 


a es dxXi,j _, capacity used 
*= "5000 = total available 
For discrete modeling, these constraints do fine. Each y; must equal 1 if corresponding 
x-variables are to use facility i at any level. In the LP relaxation, however, if x- 
variables use only a small part of the capacity, the corresponding y; will take on a 
small fractional value. 
The numerical values of Section 11.6 confirm this behavior. The LP relaxation 
of formulation (12.8) has 
yi = 0.230, 2 = 0.000, F3=0.000, 34 = 0.301 
ys = 0.115, yo =0.000, 3,=0.000, Fg = 0.650 (12.9) 
total cost = $8036.60 
with many of the y; small. 
Compare the optimal mixed-integer solution 


y=0, y=0, y3=0, y=l 
y=0, ye =, y,=0, ye=l (12.10) 
total cost = $10,153 
Bound $8036 of (12.9) is only 79% of true optimal value $10,153. Also, (12.9) suggests 
that 4 centers may be needed, while the optimum opens only 2. 
Even when a center is used only fractionally, it may fulfill the whole demand 
for some single district. Such thinking suggests valid inequalities 
any; forall) WR Ne ak ene A (12.11) 


which require that the fraction a center is opened be as great as the fraction of any 
region’s demand satisfied from the center. 
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Certainly, these inequalities satisfy validity definition 12.16 | because each is 
satisfied by every integer-feasible solution to model (12.8). Also, it is easy to ful- 
fill requirement by finding solutions to the relaxation of (12.8) that violate 
(12.11). 

Adding these valid inequalities improves the LP relaxation dramatically. The 
strengthened model has optimal solution 


3, = 0.000, j2=0.000, 73=0.000, 4 = 0.537 

Js = 0.000, Fo =0.000, ¥7=0.000, yg = 1.000 

total cost = $10,033.68 
Its bound $10,033 is almost 99% of optimal value $10,153, and only one discrete 
variable comes out fractional. Addition of the valid inequalities (12.11) has produced 


a much stronger relaxation, which provides much better information about the form 
of a discrete optimum. 


SAMPLE EXERCISE 12.11: RECOGNIZING USEFUL VALID INEQUALITIES - 
Consider the ILP 

max 3x, + 14x. + 18x 

st. 3x, +5x2 + 6x3 < 10 

X1,%2,X%3 = Oor1 

with LP relaxation optimum x = (0, z, 1). Determine (by inspection) whether each 
of the following inequalities is valid for this model, and if so, whether adding it would 
strengthen the LP relaxation. 


(a om = a < 1 


(b) x7 + x2 -+ x3 <1 
(c) 3x1 + 5x2 < 10 


Analysis: We apply definition | 12.16 | and principle | 12.17}. 


(a) It is obvious from the main constraint that no feasible solution can have both 
xX. = land x3 = 1. Thus the constraint is valid. Also, the current LP relaxation 
optimum is one LP-feasible solution that violates the inequality because 


%+ig=2+141 
It follows that the constraint will strengthen the relaxation. 


(b) This constraint is not valid. For example, x = (1, 0, 1) violates the constraint even 
though it is integer-feasible in the given model. 


(c) This constraint is valid, because any integer-feasible solution satisfying main con- 
straint 3x; + 5x2 + 6x3 < 10 certainly has 3x, + 5x2 < 10. Still, this will also be true 
of all feasible solutions in the LP relaxation. Adding the inequality cannot improve 
the relaxation. 
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Lagrangian Relaxations 


Even when the given model is an ILP, the strongest practical relaxation may not be 
the LP form obtained when integrality constraints are dropped. Lagrangian relax- 
ations, which prove stronger for some model forms, adopt a completely different 
strategy. Instead of dropping integrality requirements, they relax some of the main 
linear constraints of the model. However, the relaxed constraints are not totally 
dropped. Instead, they are dualized or weighted in the objective function with suit- 
able Lagrange multipliers to discourage violations. 


ot an ILP a moving them to the o - ctive function as terms 


22s = =e to) 


aoe vy isaLa grange multiplier chosen as the relaxation is formed. If the re- 
laxed constraint has form 2, GX) = b;, multiplier v; < 0 for a maximize model 
and v; > 0 in a minimize. If the relaxed constraint is }7; a;;xj < b;, multiplier 


fora 2 may co model and v; < 0 for a minimize ‘model. Equality con- 
: = — unrestricted ww Vj. 


Lagrangian Relaxation of the CDOT Example 
We can illustrate with the CDOT generalized assignment model (11.13) of Section 
11.4: 
min 130x11 + 460x1,2 + 4021.3 + 30x21 + 150x22 + 370X2,3 
+510x3,1 + 20x3,2 + 120x3.3 + 30x41 + 40x42 + 390Xx43 
+ 340x5,1 + 30x52 + 40x52 + 20%6,1 + 450%6,2 + 30X63 


St. x114+%12+%13 =1 (district 1) 
Moe ah 42'2) 23 (district 2) 
X31 +%32 +%33=1 (district 3) 
X41 + X42 + X43 =1 (district 4) 
X51 +%5,2 +X53 =1 (district 5) (12.12) 
x61 + %62 +%63 =1 (district 6) 
30x1,1 + 50x2,1 + 10x31 Estevan 
+ 11x41 + 13x51 + 9X61 < 50 

10x1,2 + 20x22 + 60x3,2 (Mackenzie) 
+ 10x42 + 10x52 + 17x6.2 < 50 

70x1,3 + 10x2,3 + 10x33 (Skidegate) 
+ 15x43 + 8x53 + 12x63 < 50 


xij =Oorl k= 1 6r7 = 1.3 
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where 
Ree il if district iis assigned to ship j 
Ey aoal t(D otherwise 


One strong Lagrangian relaxation dualizes the first 6 main constraints with 
weights 


v; & the Lagrange multiplier on the constraint for district 7 
The result is 
min 130x1.1 + 460x1,2 + 40x1,3 + 30x21 + 150x22 + 370x2,3 
+ 510x3,1 + 20x32 + 120x33 + 30x41 + 40x42 + 390x4,3 
+ 340x514 + 30x52 + 40x52 + 20X61 + 450%6,2 + 30X63 
+1 — x11 — X1,2 — X13) + v2(1 — X21 — ¥2,2 — X2,3) 
+3(1 — 13.4 — %3,2 — X33) + val — X41 — X4,2 — ¥4,3) (12.13) 
+5(1 — x51 — 5,2 — 5,3) + V6(1 — X6,1 — X6,2 — X6,3) 
st. 30x11 + 50x21 + 10x31 + 11x41 + 13x51 + 9x61 < 50 
10x1,2 + 20x22 + 603.2 + 10x42 + 10x52 + 17x62 < 50 
70x1.3 + 10x23 + 10x33 + 15x43 + 8x53 + 12x63 <50 
xj =Oor1 bh 6. j:—3 
Notice that the 6 equality constraints of full model (12.12) have not been com- 
pletely dropped. Instead, they have been rolled into the objective function as in 


construction | 12.18]. For example, the model (12.13) objective now includes the 
term 


++» +73(1 =X 4 — X32 — X33) +°- (12.14) 
Feasible solutions in this Lagrangian relaxation may very well have 
x31 +432 +433 41 or (L—x31 —x3,2 — x33) £0 


But if weight v3 + 0, violations will at least affect the objective value through (12.14). 


Return to Bison Boosters example model (12.2): 
max 20x; + 30x2 —550y; — 720y2 
st. 15x, + 4x2 < 300 
xy 20071 <0 
2 T5y2 <0 
Lipo = 0 
Vie Oorl 


(a) Use multipliers v; and v2 to form a Lagrangian relaxation dualizing the last two. 
switching constraints. 


(b) Indicate any required sign restrictions on multipliers v; and v2. 


12.3 Stronger LP Relaxations, Valid Inequalities, and Lagrangian Relaxations 649 


Analysis: We apply construction | 12.18 |. 


(a) The Lagrangian relaxation is formed by moving the two switching constraints to 
the objective function as 


max 20x; + 30x2 — 550y1 — 720y + vy (0 — x1 + 200y1) + ¥2(0 — x + T5y2) 
st. 15x, + 4x2 < 300 

X1,X2 > 0 

Yi, y2 =Oorl 


(b) For < constraints in a maximize model, multipliers should satisfy v1, v2 > 0. 


Tractable (Integer) Lagrangian Relaxations 
Notice that the Lagrangian relaxation (12.13) keeps variables x; j discrete. Integrality 
requirements of the original (12.12) have not been dropped. 

Lagrangian relaxations fulfill principle ’s mandate for increased tractabil- 


ity by dualizing enough linear constraints that the remaining discrete problem is 
manageable. 


12.19. | Constraints chosen for dualization in Lagrangian relaxations should leave 
a still-integer linear program with enough special structure to be relatively 
tractable. oF 


A close look at the remaining constraints in Lagrangian relaxation (12.13) will 
reveal how it conforms to requirement [12.19]. After dualization, each Xj; Occurs 
in exactly one main constraint. Thus these relaxations can be solved as a series 
of single-constraint, knapsack ILPs (definition [17.4], Section 11.2), which are the 
simplest of integer programs. There is one for each of the 3 ships. 


Lagrangian Relaxation Bounds 


Dropping linear constraints in a Lagrangian relaxation 12.18} cannot eliminate any 


solutions. That is, Lagrangian relaxations parallel property | 12.4 |in having every so- 
lution feasible in the full model still feasible in the relaxation. But Lagrangian forms 
are more complex than constraint relaxations because they modify both constraints 
and objective. Fortunately, they still yield bounds. 


12.20 The optimal value of any Lagrangian relaxation of a maximize model us- 


sign rules of definition assure that every feasible solution in the original ILP 
echieves no less in the objective function of the Lagrangian relaxation when we 
maximize and no more when we minimize. Thus the relaxation optimum, which 
=ither equals or improves on all these results for truly feasible solutions, must yield 
2 bound. 
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Choosing Lagrange aa 


How strong the bounds of 12.20) .20 | prove to be in any Lagrangian relaxation depends 
on the multiplier values chosen. Some choices of v; will produce a very weak La- 
grangian relaxation, and others can make it quite strong. 


Methods for determining good Lagrange multipliers are beyond the scope oF 
this book, but we can illustrate their potential power with 


y= 300, y2= 200, v3= 200, VW= 4S, v5= 45, vo= 30 


in our CDOT example. The corresponding optimal value in Lagrangian relaxation 
(12.13) is $470,000, which is much closer to the integer optimal value of $480,00 
than is the LP relaxation bound $326,100. 


Return to the Bison Boosters Lagrangian Telawuion of Sariple BNerese 12.12. Sols = 
(by inspection) the relaxation for each of the following choices of Lagrange mult- 
pliers, and comment on the strength of the results. 


(a) 4) =0, v2 =1 

(b) vy = 3, v2 =3 

Analysis: 

(a) For v; = 0, v2 = 1, the Lagrangian relaxation reduces to 


max 20x, + 29x2 — 550y; — 645y2 
st. 1.5x, + 4x2 < 300 

Kite 0 

y1,¥2 = Oor 1 


With only 0-1 constraints enforced on the y;, setting ¥) = y2 = 0 produces a re 
laxation optimum. Corresponding optimal choices for the continuous variables are 
X, = 200 and X2 = 0. The resulting relaxation bound of $4000 is weak because the 
mixed-integer optimal value is $3450. 


(b) For v; = v2 = 3, the Lagrangian relaxation is 
max 17x; + 27x2 + 50y; — 495y2 
s.t. 1.5x, + 4x2 < 300 
X1,%2 2 0 
V1, ¥2 =Oorl 
With only 0-1 constraints enforced on the y;, setting ; = 1, ¥2 = 0 produces = 
relaxation optimum. Corresponding optimal choices for the continuous variables 
are again X, = 200, X, = 0. This time, the resulting relaxation bound of $3450 is ver 
strong because it exactly matches the true mixed-integer value. 
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12.4 Branch and Bound Search 


12.4 BRANCH AND BOUND SEARCH 


Total enumerations of Section 12.1 are impractical for all but the simplest models 
because every one of an explosively growing number of discrete solutions must be 
considered explicitly. The process would become much more manageable if we could 
deal with those solutions in large classes, determining for each whole class whether 
it is likely to contain optimal solutions, and doing so without explicit enumeration 
of all its members. Only the most promising classes would have to be searched in 
detail. 

Branch and bound algorithms combine such a partial or subset enumeration 
strategy with the relaxations of Sections 12.2 and 12.3. They systematically form 
classes of solutions and investigate whether the classes can contain optimal solutions 
by analyzing associated relaxations. More detailed enumeration ensues only if the 
relaxations fail to be definitive. 


EXAMPLE 12.2: RIVER POWER 


As with so many other topics, an artificially small example will aid in our develop- 
ment of branch and bound ideas. Here we consider an operations problems at River 
Power Company. 

River Power has 4 generators currently available for production and wishes to 
decide which to put on line to meet the expected 700-megawatt peak demand over 
the next several hours. The following table shows the cost to operate each generator 
(in thousands of dollars per hour) and their outputs (in megawatts). 


Generator, j 
1 2 3 4 


$$$ _ te 
Operating cost a eal 2. 5 14 
Ouput power 300 600 500 1600 
Se ee Ea nea sete 


Units must be completely on or completely off. 
We can formulate River Power’s problem as a knapsack problem like those of 
Section 11.2. Decision variables 


Ke|hedll if generator j is turned on 
i 6 
ET X0) otherwise 
Then a model is 
min 7x, + 12x2 + 5x3 + 14x4 (total cost) 
st.  300x, + 600x2 + 500x3 + 1600x4 > 700 (demand) (12.15) 


x1, X2,x3,X4 =Oorl 


the objective function minimizes total operating costs, and the main constraint as- 
sures that the chosen combination of generators will fulfill demand. Total enumer- 
‘tion establishes that an optimal solution use generators 1 and 3 and cost $12,000. 


Partial Solutions 


“uch like the improving searches of most of this book, branch and bound searches 
‘erate through a sequence of solutions until we are ready to conclude optimality or 
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stop with the best fully feasible solution found so far. What is new is that branch ans 
bound searches through partial solutions. 


22) 4 partial solution 
er undetermined. We den 


For example, in the River Power model (12.15), x = (1, #, 0, #) specifies a parts 
solution with x; = 1, x3 = 0, while x2 and x4 remain free. 


Completions of Partial Solutions 
Each partial solution implicitly defines a class of full solutions called its completions 


Ue completions of a partial solution to a given model are the possible 
full solutions agreeing with the partial solution on all fixed components. 


For instance, the completions of partial solution x = (1, #, #, 0) in our River Pow 
model are 


(15050, 0), C1 OS 0) = GL, 1 OO) rare Cle i 1) 


Every solution with x; = 1 and x4 = 0 is among these 4. The last 3 are feasiie 
completions because they satisfy all constraints of model (12.15). 


SAMPLE EXERCISE 12.14: UNDERSTANDING PARTIAL SOLUTIONS 
L AND. COMPLETIONS 


Suppose an integer program as decision variables x1, x2,.x3 = 0 or 1. Tae all ce 
pletions of each of the following partial solutions. 


(a) G #, #) 
(b) (1, #, 0) 


Analysis: We apply definitions | 12.22 | and| 12.23 |. 


(a) Completions of this partial solution consist of all full solutions with x; = 1 [< 
(1,0, 0), 0; 1), Gi, 1,0), anda 1, 1]. 


(b) Completions of this partial solution consist of all full solutions with x; = 1 = 
x3 = 0 [ie., (1, 0, 0) and (1, 1, 0)]. 


Tree Search 


Branch and bound investigates classes of solutions corresponding to completions 
partial solutions in a treelike fashion that gives it the “branch” part ofitsname. Figo™ 
12.2 provides a full example for River Power model (12.15). Nodes of this branch amt 
bound tree represent partial solutions, with numbers indicating the sequence in wh 
they are investigated. Edges or links of the tree specify how variables are fixed inp» 
tial solutions. For example, partial solution x in the sequence has x4 = 0 and x2 = 


12.4 Branch and Bound Search 


© = (0,0, 0, 0.44) 
¥=6.12 


x = (0, 0, 0, 1) 
v=14 


¥) = (0, 0.33, 1, 0) 
v=9 


by solving 
v=14 


x) = (0, 1, 0.2, 0) 
v= 13 


x = (0,67, 0,1, 0) 
Vv = 9.67 


x = (0, 1, 1,0) ¥©) = (0.33, 1, 0, 0) 
vV=17 V = 14.33 
by bound by bound infeasible 


x) = (1, 0,1, 0) 

v=12 

by solving infeasible 
v=12 


MIGURE 12.2, Branch and Bound Tree for River Power Example 
The process begins at root node 0. 


12.24 | Branch and bound search begins at initial or root partial solution xO oe 
GG. -#) with all variables free, 


This provides the first active or unanalyzed partial solution. 

At any stage of the search one or more active nodes (distinguished by not yet 
having a number) remain in the tree. Analysis of each node or partial solution at- 
tempts to decide which, if any, completions warrants consideration as an overall opti- 
mal solution. Sometimes we can either find a best completion or conclude that none is 
worth further investigation. Then we terminate or fathom the entire class of (comple- 
tion) solutions represented by the node. That is, we give it no further consideration. 


fy either any a fest completion or prove that none can produce an opti: 
mal solution in the overall model. _ 


Node 1 in Figure 12.2 illustrates termination. It has no subsidiary nodes be- 
cause analysis of partial solutionx™ = (#, #, #, 1), which we detail below, established 
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that the best possible completion is x = (0,0, 0, 1). No further investigation of any 
solution with x, = 1 is required. 

Observe that this termination dealt, in a single step, with fully half the possible 
solutions to the River Power model. That is, we enumerated the half of all solu- 
tions with xy = 1 as a class. In this way, the exponentially growing effort to totally 
enumerate every member was avoided. 

Unfortunately, it often happens that analysis is not definitive. In such cases the 
node or partial solution must be branched. 


Node 2 of Figure 12.2 illustrates the need for branching. Analysis (detailed be- 
low) was unable either to find the best completion of partial solution x?) = (#, #, #, | 
or to prove that none could be optimal. Thus the node was branched into those num- 
bered 3 and 6. Both have x4 = 0, as in the x. However, previously free variable 
x2 has now been fixed. In partial solution 3, it is fixed = 1. In partial solution 6, it? 
fixed = 0. 

Notice that this branching process loses no solutions. Every completion of nods 
2 has either x7 = 0 or x7 = 1. We have simply constructed 2 smaller classes of solu- 
tions in the hope that our analysis will now be strong enough to permit termination 

Since no solutions are lost, the enumeration is complete when all partial solu- 
tions have been resolved definitively. 


As long as partial solutions do remain, branch and bound search must selee 
an active one to explore next. The simplest such scheme is known as depth first. 


The River Power enumeration of Figure 12.2 employs this depth first rule. Fo 
example, after node 3 had been explored, the partial solutions corresponding * 
nodes 4, 5, and 6 were active in the tree. In accord with depth first rule , OnE 
of the deeper nodes 4 and 5 was selected to investigate next. 


The Paras is the branch and bound tree for a discrete SptMuOH model wi 
decision variables x1, x2,.x3 =Oor1. 


SS) a, ee Ly ee an ee ne ae  U— 
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_ (a) List the sequence of partial solutions explored. 


(b) Identify which of the partial solutions in part (a) were terminated and which 
branched. 


(c) Determine which nodes would have been active just after processing of node 1, 
and explain which could have been the next explored under the depth-first enumer- 
ation rule. 


(d) Demonstrate that all possible solutions were implicitly enumerated by specifying 
for each the node at which it was terminated. 


Analysis: 


(a) Under principle , the first partial solution x of a branch and bound search 
is always the all-free (#, #, #). From variable restrictions on branches, we can see sub- 
sequent partial solutions visited were x = (#, 1, #),x® = (#,0,4,x® = (1,1, #), 
and x = (0,1, #). 


(b) A lack of subsidiary nodes shows that partial solutions 2, 3, and 4 were terminated. 
Remaining nodes 0 and 1 were branched. 


(c) After processing node 1, 3 active partial solutions remained in the tree. None 
would then have had a number, but they eventually became node 2 created when 
partial 0 was branched, together with nodes 3 and 4 formed at 1. Depth first enumer- 
ation would have taken the search to either of the deeper nodes 3 and 4. However, 
the search in this example proceeded, instead, to node 2. 


d) A full solution is implicitly enumerated when a completion class to which it be- 
longs is terminated. The following table shows the node at which each of the 8 
solutions was terminated in this case. 


Solution Node | Solution Node 
(0, 1, 0) 4 (1, 0, 0) 2) 
(0,1, 1) 4 (1,0, 1) D 


Incumbent Solutions 


Whether explicit or implicit, the goal of any enumeration is to identify an optimal 
or at least a good feasible) solution to some optimization model. To that end, it is 
essential to keep track of the best known or incumbent solution. 


Node Solution Node 


(1, 1, 0) 
(1, 1, 1) 


Solution 


(0, 0, 0) 
(0,0, 1) 
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12.29 2 The incumbent. solution at ey stage ina ee of a discrete model is the 


The incumbent solution may derive from experience prior to the search, or it m: 
have been discovered as the search evolved. 

When the search stops, the last incumbent solution is its output. Assum= 
that the given model has an optimal solution, the final incumbent at least provics 
an approximate optimum, %, with corresponding incumbent solution value ). If 
search was fully carried out, the optimum is exact. 


12.30 | Ifa branch and bound search stops as in| 12,27 |, with all partial solutions 
having been either branched or terminated, the final incumbent solution is a 
global optimum if one exists. Otherwise, the model is infeasible. 


SAMPLE EXERCISE 12.16: UNDERSTANDING INCUMBENT SOLUTIONS 


Return to the branch and bound tree of Sample Exercise 12.15 and assume (i) 1% 
we were maximizing, (ii) that from prior experience we knew a feasible solution » 
objective value 10, and that (iii) analysis leading to terminations at nodes 2, 3. = 
4 showed the best feasible completions of the corresponding partial solutions he 
objective values 8, 14, and 12, respectively. 


(a) Show the sequence of incumbent solution objective values. 
(b) Determine the optimal solution value. 
Analysis: 


(a) From assumption (ii) the initial incumbent solution value would have been ) = 
This value held until node 3 because node 1 was branched and the best complet = 
of node 2 did not improve on 10. At node 3, the search uncovered a feasible solu» 
with better value 14, so this became the incumbent solution value to ) = 14. Nog= = 
produced no change. 


(b) This search implicitly enumerated all possible solutions. Thus, by the prince = 
12.30 |, the final incumbent solution value > = 14 is the optimal value. 


Candidate Problems 


Having introduced the tree search underlying branch and bound, we are now ress 
to see how relaxations of Section 12.2 and 12.3 can make it efficient (and Justify 
“bound” part of its name). Candidate problems provide the linkage. 


“231 | The candidate problem associated with any partial solution to an opti- 
mization model i tricted version of the model obtained when variables 
are fixed as in the pa 


12.4. Branch and Bound Search 657 


We may illustrate with partial solution x® = (#, 1, #, 0) in River Power Figure 
12.2. The corresponding candidate problem is 


min 7x; + 12x, + 5x3 + 14x4 

st. 300x; + 600x2 + 500x3 + 1600x4 > 700 
x1,x%3 =Oorl 
x2 = i x4 = 0 


It derives from original model (12.15) by restricting x2 and x, to their partial solution 
values. 

Thinking about candidate problems aids branch and bound search because of 
the close connection with completions of the corresponding partial solution. 


“problem. 


That is, we can search for a best completion of any partial solution, or at least learn 
something about a solution’s objective value, by trying to optimize the corresponding 
candidate problem. 


Consider the ILP 
max 10w; +3w2+4+9w3 
s.t. 6w, + 4w2 + 3w3 < 10 
W2 —w3 > 0 
W1,W2,W3 = Oor 1 
(a) State the candidate problem corresponding to partial solution w = (1, #, 0). 


(b) State the LP relaxation of the candidate problem corresponding to partial solution 
w = (1, #, 0). 


Analysis: 


(a) Following definition | 12.31), the required candidate problem is the restricted 
version with w; = 1, w3 = 0: 


max 10w; +3w2 + 9w3 

s.t. 6w, + 4w2 + 3w3 < 10 
W2 — w3 > 0 
Wy= ile wWw3 = 0 
w2 =Oorl 


An optimal solution provides a best completion of the partial w = (1, #, 0) (principle 


1232). 


b) Applying definition , the LP relaxation of the candidate problem in part 
a) is 
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max 10w; +3w2 + 9w3 

s.t. 6w) + 4w2 + 3w3 < 10 
wW2—-w3>0 
Wyi= ils wWw3= 0 
0O<w2<1 


Notice that it is a relaxation of the candidate problem in part (a), not a relaxation of 
the full model. 


Terminating Partial Solutions with Relaxations 


Now we are ready to take advantage of the relaxation principles in Section 12.2. We 
analyze nodes in a branch and bound tree by solving relaxations of the corresponding 
candidate problems. For example, relaxation solutions and solution values shown 
next to the nodes in River Power Figure 12.2 (denoted x) come from the linear 
programming relaxation of the corresponding candidate problem. 

Begin with infeasibility principle . If any relaxation of a candidate prob- 
lem is infeasible, so is the full candidate. 


i205 | Ifany relaxation of a candidate problem proves infeasible, the associated 
partial solution can be terminated because it has no feasible oe a 


Node 10 of Figure 12.2 illustrates. The LP relaxation of the corresponding 
candidate problem is infeasible. It follows that partial solution x“ = (0, 0, #, 0) has 
no feasible completions, and we may terminate infeasible. 

Now consider relaxation bound principle [12.9]. Relaxation optimal values 
bound those of the problem relaxed. Thus in the context of candidate problems. 
relaxation optimal values bound the objective value of the best possible comple- 
tion. Comparison to the incumbent solution value (definition /12.29]) can lead to 
termination. 


12.34 : If any relaxation of a candidate problem h 
better than the current incumbent solution value : 
tion can be terminated because no feasible compl tion can improve on the 
incumbent. . 


Node 5 of Figure 12.2 illustrates such termination by bound. When the search 
reached partial solution x® = (#,1, 0,0), incumbent solution x = (0, 0, 0, 1) with 
objective value } = 14 was already in hand (from node 1). The linear programming 
relaxation of the candidate problem for node 5 had optimal value 14.33, which means 
that no feasible completion can do better than that value. It follows that none can 
improve on the incumbent solution in this minimizing problem, and we terminate by 
bound. 

The third way to terminate with relaxations derives from optimality principle 


[12.10]. 


12.4 Branch and Bound Search 


_12.35 | If an optimal solution to any constraint relaxation of a candidate problem 
is feasible in the full candidate, itis a best ee eee of _ associated _ 


Consider node 1 of River Power Figure 12.2. The corresponding LP relaxation 
fixed x4 = 1 but allowed free variables to take on any value between 0 and 1. Still, re- 
laxation optimum x = (0, 0, 0, 1) meets integrality requirements on all components. 
It is optimal in the corresponding candidate problem and the best possible comple- 
tion of partial solution x” = (#,#,#,1). It is also the first fully feasible solution 
encountered in the search, so it provides the first incumbent solution. After saving 
it as the incumbent, node 1 was terminated by solving. 


MPLE EXERCISE 12.18: TERMINATING PARTIAL SOLUTIONS _ 
WITH RELAXATIONS 


Suppose that a maximizing branch and bound search over y;,..., ya = 0 or 1 reaches 
partial solution y®) = (#, 0, #, #) with incumbent solution value / = 100. Explain 
how the search should proceed assuming each of the following outcomes from an 
attempt to solve the linear programming relaxation of the corresponding candidate 
problem. 


(a) Relaxation optimum y = ( i, 0, 1, 0) with objective value ) = 85, 
(b) Relaxation optimum y = (1, 0, 5. 0) with objective value ) = 100. 
(c) Relaxation optimum y = (0, 0, 1, 1) with objective value i = 120. 
(d) Relaxation infeasible. 


(e) Relaxation optimum y = (0, i 1, 0) with objective value b = 111. 


Analysis: 


(a) The relaxation bound demonstrates no feasible completion can do better than 85 
in objective value, which is worse than the known incumbent. The partial solution 


should be terminated as in principle | 12.34}. 


(b) The relaxation bound demonstrates that no feasible completion can do better 
than 100 in objective value, which is the same as the incumbent. The partial solution 


should be terminated as in principle | 12.34 | (unless alternative optimal solutions are 
of interest). 

(c) This relaxation optimum is feasible in the full candidate problem and thus optimal. 
Having found the best possible completion, we terminate by principle | 12.35 | after 
updating the incumbent solution value to improved value } <— 120. 


(d) There are no feasible completions. The partial solution should be terminated as 


in principle : 
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(e) Here none of principles | 12.33 | to | 12.35 | lead to termination because the relax- 
ation optimum is better in objective value than the incumbent but still fractional in 


LP-Based Branch and Bound 


The discrete models most frequently solved by branch and bound are ILPs with 0-1 
variables. Linear programming relaxations of candidate problems usually provide 
the basis for analysis. 

Algorithm 12A details a formal algorithm for this LP-based branch and bound 
case. For simplicity we assume that the model is not unbounded. 


ALGORITHM TA: LP-BASED BRANCH AND BOUND (0 


)-1ILPS) 


Step 0: Initialization. Make the only active partial solution the one with all 
discrete variables free, and initialize solution index t < 0. If any feasible 
solutions are known for the model, also choose the best as incumbent solution 
& with objective value 6. Otherwise, set § <— —oo if the model maximizes 
and } < +00 if it minimizes. 

Step 1: Stopping. If active partial solutions remain, select one as x, and 
proceed to Step 2. Otherwise, stop. If there is an incumbent solution X, it is 
optimal, and if not, the model is infeasible. 

Step 2: Relaxation. Attempt to solve the linear programming relaxation of 
the candidate problem corresponding to x. 

Step 3: Termination by Infeasibility. If the LP relaxation proved infeasible, 
there are no feasible completions of partial solution x. Terminate x, 
increment t <— t+ 1, and return to Step 1. 

Step 4: Termination by Bound. If the model maximizes and LP relaxation 
optimal value * satisfies i < }, or it minimizes and > > 6, the best feasible 
completion of partial solution x cannot improve on the incumbent. Terminate 
x, increment t < t+ 1, and return to Step 1. 

Step 5: Termination by Solving. If the LP relaxation optimum x satisfies 
all binary constraints of the model, it provides the best feasible completion 
of partial solution x. After saving it as new incumbent solution 


Ki x 
DoD 
terminate x, increment t — t+ 1, and return to Step 1. 

Step 6: Branching. Choose some free binary-restricted component x, that 
was fractional in the LP relaxation optimum, and branch x by creating two 
new actives. One is identical to x except that x, is fixed = 0, and the other 
the same except that x, is fixed = 1. Then increment ¢ <— t+ 1 and return 
to Step 1. 


Initialization begins as in principle | 12.24 | with all binary variables free. If no 
incumbent solution x is known, the worst possible value ) < too is assumed. 


(oo SO ee SS 


12.4 Branch and Bound Search 


Each main iteration begins by selecting some active partial solution to pur- 
sue. Any one can be selected, although sequence does make a difference (see Sec- 
tion 12.5). 


Processing begins with an attempt to solve the LP relaxation. Then we check 


to see if any of termination rules | 12.33 | to | 12.35] apply. If so, the current partial 
is terminated and the process repeats. Partial solutions that cannot be terminated 


must be branched (principle | 12,26 |). 


Branching Rules for LP-Based Branch and Bound 


One peculiarity of the LP-based form of branch and bound is that this is done on a 
fractional free variable. 


6 LP-based branch and bound algorithms always branch by fixing an integer- 
testricted decision variable that had a fractional value i in the associated candi- 
date problem relaxation. 


For example, at node 0 of in Figure 12.2, branching occurred by fixing x4, which had 
fractional value X4 = 0.44 in the LP relaxation. 

The motivation behind this fractional variable branching rule is avoiding du- 
plicate computation. The alternative of branching on a free variable that came out 
integer in the LP relaxation produces one new candidate problem guaranteed to 
have the same relaxation optimum as the one just solved. For example, branching 
on x; at node 0 of Figure 12.2 would have created new partial solutions 


x) = (1,#,#,#) and x? = 0,#,#,#) 


But the relaxation optimum for the second would be exactly the same as * = 
(0, 0, 0, 0.438) because added constraint x; = 0 can affect solution value only if it 
is violated by the previous optimum. We prefer to get new information from both 
candidates. 

Somewhat similar thinking applies when more than one integer variable is 
fractional in the relaxation optimum. 


12, 31 When more than one antes -restricted variable is is s fractional in the relax- 


For example, if the relaxation optimum were x = (0.3, 1, 0.5, 0.9), and all components 


were supposed to be binary, rule | 12.37 | would next fix x4 because it is the fractional 
variable closest to an integer. 


The motivation behind rule| 12.37 | is to take the most obvious decision, hoping 
that the farthest away of the two resulting partial solutions can be terminated before 
it has to be explored. Still, many other schemes are possible. Commercial codes 
»iten give the modeler an opportunity to explicitly designate priority variables for 
oranching. 
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LP-Based Branch and Bound Solution of the River Power Example 


We are now ready to fully trace Figure 12.2’s branch and bound solution of River 
Power model (12.15). 

The process begins with all-free partial solution x© = (#, #, #, #) and ) = +c 
The corresponding LP relaxation optimum is fractional on x4, so the node is branched 
(rule | 12.36 |) as 


¥ = (0, 0, 0, 0.438), ¥ = 6.125 


There are now 2 active partial solutions, and we must choose one to process 
next. Any active partial solution could be chosen, but all Figure 12.2 computation 
employs the depth first rule 12.28). That is, it chooses an active partial solution 
deepest in the tree. 

At the moment both active candidates have equal depth because both have | 
fixed variable. Figure 12.2 adopts the simple tie-breaking rule of always selecting the 


partial solution with the branching variable fixed = 1. (See rule |1 45 of Section 
12.5 for another possibility.) 

That tie-breaking rules leads us to x = (#, #, #, 1). Here the relaxation solv- 
tion is binary on all components. Thus we have found the best completion, and we 


terminate (rule | 12.35) after saving incumbent solution 
x <(0,0,0,1) with }< 14 
The only remaining active is now partial solution x™ = (#, #, #, 0). Relaxation 


optimum x” = (0, 0.333, 1, 0) fails all termination tests to}1 because it 
is feasible, fractional, and has solution value 9 strictly better than } = 14. We mus? 
branch on fractional x2. 

Processing continues in this way until we reach x“ = (#,1,1,0). There the 
relaxation bound ) = 17 > > = 14, so we terminate (rule | 12.34]). Something 
similar happens at node 5. 

After several more steps, we encounter a new incumbent solution at node 8. 


x < (1,0,1,0) with }<12 


The WHEE table shows candidate wvoblent LP <elaiGn optima for all possible 
combinations of fixed and free values in a maximizing mixed-integer linear program 
over x1, X2,x3 =Oorl, x, >0. 


12.4 Branch and Bound Search 


XX x Vv XX 2X3 x Vv 
# # # (0.2,1,0,0) 8280] 0 0 1 Infeasible — 
# = © 2 = (O25 15050) 5822805 10 1 # = (0,1,0.67,0) 80.67 
# # 1 (0,0.8,1,0) 79.40 | 0 1 0 (0, 1, 0, 2) 28.00 
# (0) #  (0.7,0,0,0) 81.80 | 0 1 1 (0,1,1,0.5) 77.00 
# 0 0  (0.7,0,0,0) 81.80 | 1 # # (1, 0, 0, 0) 74.00 
# 0 1 (0.4,0,1,0) 78.60 | 1 # 0 (1, 0, 0, 0) 74.00 
# at # (0.2,1,0,0) 82.80 | 1 # 1 (1, 0, 1, 0) 63.00 
# 1 O (0.2,1,0,0) 82.80} 1 0 # (1, 0, 0, 0) 74.00 
# 1 1 (0,1,1,0.5) 77.00 | 1 0 0 (1, 0, 0, 0) 74.00 
(0) # # (0,1,0.67,0) 80.67 | 1 0 1 (1, 0, 1, 0) 63.00 
0 # 0 (0, 1, 0, 2) 28.00 | 1 al # (1, 1, 0, 0) 62.00 
(0) # 1 (0, 0.8,1,0) 79.40 | 1 1 0 (1, 1, 0, 0) 62.00 
0 0 # Infeasible —_— 1 1 ib (@5151;;0) 51.00 
0 0 0 Infeasible — 


Solve the model by LP-based branch and bound Algorithm 12A, applying the same 
depth first rule for selecting among actives (ties broken in favor of x; = 1) that was 
implemented in the River Power example of Figure 12.2. 


Analysis: Applying Algorithm 12A produces the following branch and bound tree: 


*¥ = (1, 0,0,0), = 74 


by solving 


v=74 x3=1 


@ x = (0,1, 1, 0.5), 
ow Vv=T7 
by solving 

v=T77 


infeasible 


x) = (0.2, 0.1, 0, 0), 7 = 82.8 


x3 =0 


x@) = (0, 1, 0.67, 0), ¥ = 80.67 


x = (0,1, 0,2), = 28 


Processing stops with optimal solution x* = (0, 1, 1, .5) at objective value 77. 
Most of the processing is similar to our River Power example. One exception 
is that this model maximizes. For example, we terminate by bound at node 6 because 


BH=vsv=77 


The other new element is that this model has continuous variable x4 as well 
2s 3 binary ones. This makes relaxation optimum x® = (0,1, 1, 0.5) feasible (and 
so optimal) in the full candidate, even though its last component is fractional. We 


terminate by having solved the candidate problem. 
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12,5 ROUNDING, PARENT BOUNDS, ENUMERATIONS 
SEQUENCES, AND STOPPING EARLY IN BRANCH 
AND BOUND 


Algorithm 12A contains all the main elements of at least LP-based branch and boune 
but it omits many details. In this section we briefly introduce some refinements. 


Branch and Bound Solution of NASA Capital Budgeting Example 


It will help to have a more serious example to illustrate. For this purpose we emplo" 
the NASA mission selection model formulated in Section 11.2. There, the decision 
variables 


ie 1 if mission j is selected 
J) (0) otherwise 


The complete formulation of our version was 
max 200x; + 3x2 + 20x3 + 50x4 + 70x5 
+ 20x65 + 5x7 + 10xg + 200x9 + 150x109 
+ 18x11 + 8x42 + 300x13 + 185244 
s.t. 6x, + 2x2 + 3x3 + 1x7 + 4x9 + 5x42 


(total value) 


(2000-2004) 


3x2 + 5x3 + 5x5 + 8x7 + 5x9 + 8x19 

+ 7x12 + 1x43 + 4x44 
8x5 + 1x6 + 4x19 + 2x11 + 4x13 + Sx14 
8x6 + Sxg + 7x11 + 1x13 + 3x14 


(2005-2009) 


(2010-2014) 
(2015-2019) 


10x4 + 4x6 + 1x13 + 3x14 <14 (2020-2024) 

xgt+x5 <1 (mutually Oe 
Ca ee exclusives) 

Xo +xy4 <1 

X41 < x2 (dependent 

een missions) 

x5 SX3 

x6 5X3 

x7 5X3 


x;=Oorl forall j=1,...,14 
Figure 12.3 shows the tree of an LP-based branch and bound search, and Table 12 
details computations. 


Rounding for Incumbent Solutions 
One refinement of branch and bound exhibited in these NASA example computs- 


tions is rounding relaxation optima as in principle | 12.11] to speed the process « 
finding good incumbent solutions. 
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¥ = 826.250, X19 = 0.875, round = 695, V = 695 


x19 = 1 x49 = 0 
V = 798.750, ¥14 = 0.75, = V = 793.171, ¥3 = 0.78, 
round = 660 round = 510 
X44 =0 %3=1 x3 =0 
V = 780, Xo = 0.60, (4) V = 791.250, Xo = 0.25, Vase l.d 
round = 660 round = 540 
by bound by bound 
Xg=l X99 = 0 Xj9=1 Xo =0 
Ge Bs V = 787, X4 = 0.60, 
: Mott G1) v = 704 (5) round = 725, ¥ =725 
infeasible by bound by bound 
X4 = 1 X4= 0 
v= 770.5, 2 = 0.50, G2) 7 = 757, 
round = 755, 0 = 755 5 = = 0.40, 
round = 725 
X,=1 %2=0 Xs =1 x5 =0 
V = 766.905, 
X7 = 0.09, 
G) %= 700.167 (5) round = 765, @) v= 732 
by bound Rue by parent by bound 
after 15 
x7 =1 x,=0 
Gs) 6) V = 766.600, X12 = 0.20, 
round = 765 
infeasible 
x2=1 x2 = 0 
¥ = 174.667 Q7) ¥ = 765 
by bound 


by bound 


FIGURE 12.3. Branch and Bound Search of NASA Example 
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TABLE 12.3. Branch and Bound Search of NASA Example 
Relax Relaxation Round 
t Value Solution Value Action 
0 826.250 (1, 0, 0, 0, 0, 0, 0, 1, 0, 0.875, 0, 0, 1, 1) 695 First incumbent & << 
(1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, { 
branch on X49 
1 798.750 (1 ,0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0.750) 660 Branch on x14 
2 545.000 (1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1) — Terminate by bound 
3 793.171 (1, 0, 0.780, 0.463, 0.537, 0.780, 0, 1, 0.415, 0, 0, 0, 1, 0.585) 510 Branch on x 
4 791.250 (1, 0, 1, 0.650, 350, 1, 0, 0.550, 0.250, 0, 0, 1, 0.750) 540 Branch on X» 
5 787.000 (1, 0, 1, 0.600, 0.400, 1, 0, 0.400, 0, 0, 0, 0, 1, 1) 725 New incumbent x < 
(1, 0, 1, 0, 0, 1, 0, 0, 0, 0,0, & 
branch on x, 
6 770.500 (1, 0.500, 1, 1, 0, 0, 0, 0.500, 0, 0, 0.500, 0, 1, 1) (ES) New incumbent x <— 
(1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0. ! 
branch on x, 
7 700.167 (0.833, 1, 1, 1,0.188, 0, 0, 0, 0, 1, 0, 1, 0.750) — Terminate by bound 
8 780.000 (1, 0, 0, 0, 0, 0, 0, 1, 0.600, 1, 0, 0, 1, 0) 660 Branch on xo 
9 Infeasible None — Terminate by infeasible 
10 713.500 (1, 1, 0, 0, 0, 0, 0, 0, 0.500, 0, 1, 0, 1, 0.500) — Terminate by bound 
11. 704.000 (0.500, 0, 1, 0.800, 0.200, 1, 0, 1, 1, 0, 0, 0, 1, 0) — Terminate by bound 
12. 757.000 (1, 0, 1, 0, 0.400, 1, 0, 0.400, 0, 0, 0, 0, 1, 1) 725 Branch on x, 
13 732.000 (1, 0.462, 1, 0, 0, 0.846, 0, 0.077, 0, 0, 0, 0.462, 0, 1, 1) — Terminate by bound 
14 =714.000 (1, 0, 0.600, 0.600, 0, 0.600, 0, 0, 1, 0, 1, 0, 0, 1, 0) — Terminate by bound 
15 766.909 (1, 0, 1, 1, 0, 0, 0.091, 1, 0, 0, 0, 0.182, 1, 1) 765 New incumbent X <— 
(1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, ' 
branch on x7 
16 766.600 (1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0.200, 1, 1) 765 Branch on x, 
17 765.000 Ges 1, 100705 1, 050, 0, 0;4,1) — Terminate by bound 
18 Infeasible None — Terminate by infeasible 
19 174,667 (0,333, 0, 1, 1,0, 1, 0, 1, 0, 0, 0, 1, 0, 0) — Terminate by bound 


ep 


“ Underlined valucs arc fixed in the partial solution. 


Computations in NASA Table 12.3 round down; that is, fractional components 
in relaxation optima are set = 0. Every partial solution that cannot be terminated 
is rounded before branching, and the resulting feasible solution considered as an 
incumbent. For instance, at node 5, the candidate relaxation produced optimum 


x = (1,0, 1,0.6,0.4, 1, 0,0.4,0,0,0,0,1,1),  +=787 


The solution is fractional, and bound 787 is insufficient to terminate. Instead of 
branching immediately, however, the relaxation is first rounded by setting each x; = 
|x; ]. The result is new incumbent solution 


si=161,'0)-1,0,'0, 1.0, 07070;070,.1,-1), PS 


Rounding is not guaranteed always to produce a new incumbent. For example. 
at node 1 of the NASA example, rounding produced a solution of value 660, which 
was no better that the existing incumbent b = 695. 

It is valuable to have good incumbent solutions early in a branch and bound 
search, for two reasons. First, time may not permit the search to be run until every 


— 
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node has been terminated or branched. Then the final incumbent solution provides 
an approximate optimum. Clearly, we would like it to be as good as possible. 
The other advantage of early incumbents comes with termination by bound 


(principle 12.34 ). In the maximize case, for example, we terminate if 


relaxation bound 4 >) < >A incumbent value 


We can certainly terminate more nodes with the same bound values if a strong in- 
cumbent value is discovered quickly. 


SAMPLE EXERCISE 12.20: ROUNDING FOR INCUMBENT SOLUTIONS 


The following tree shows Algorithm 12A computation on a minimizing ILP over 
X4,...,X3 = Oor1. (Rounded solution values are shown but were not used in the 
computation.) 


X = (0, 0.5, 1), 7 = 10 
round = (0, 0, 1), value 14 


¥ = (0.7, 1,0), 7 = 20 


round = (1, 1, 0), value 32 ca AUT alee 


infeasible 


_ (a) Determine the earliest moment in the search at which ultimate optimal solution 


value 14 was known. 


(b) Repeat the search using rounding to produce earlier incumbent solutions. 
(c) Describe the savings that resulted in part (b) from using rounded solutions. 
Analysis: 


(a) The final incumbent solution, which proved optimal, was encountered when the 
relaxation optimum at partial solution 4 solved its candidate problem. 


(b) Atnode 0 the relaxation optimum has value 10, and it rounds to a feasible solution 
with value 14. This becomes the first incumbent solution value. We then branch on 
xX as before. At node 1 relaxation bound 20 permits termination because it does not 


improve on the incumbent (principle | 12.34/). Then the relaxation at node 4 the 
verifies optimality of the incumbent solution. 


(c) In the search of part (b) the incumbent was uncovered at node 0, much earlier 
than node 4 in calculations depicted in the tree. This would be an advantage if we 
had to stop the search before all nodes had been terminated or branched. Also, the 
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early incumbent avoided some candidate problem computations at nodes 2 and 3 
The bound at 1 is now enough to terminate. 


Branch and Bound Family Tree Terminology 


To discuss issues connected with managing and controlling branch and bound trees 
we need some terminology. It is standard to make analogies with family trees. 

Any node created directly from another by branching is called a child, and the 
branched node is its parent. For example, in Figure 12.3, nodes 11 and 5 are the 
children of node 4. Its parent is node 3. 


SAMPLE EXERCISE 12. 21: UNDERSTANDING TREE TERMINOLOGY : 

Identify each of the following in the branch and bound tree of Sample Exercise 12.2! 
(a) The parent of node 3 

(b) The children of node 0 


Analysis: 
(a) Node 1 is the parent of node 3. 
(b) Nodes 1 and 4 are the children of node 0. 


Parent Bounds 


Another way of refining LP-based branch and bound Algorithm 12A is to take ad- 
vantage of an easy parent bound readily available from prior computations. 


model provides an ‘upper bound. — 


To see that these bounds are valid, recall that the candidate problem associated 
with any partial solution is just our original model augmented by constraints for 


variables fixed in the partial solution (definition [12.31]). Extending the solution 
with a new fix in a child can only worsen the optimal objective function value. For 
example, in Figure 12.3, relaxation bound 826.250 for node 0 holds for its childrez 
nodes 1 and 3. Both have an additional constraint, so the solution value can only 
worsen. In fact, when relaxation bounds ) = 798.750 and )® = 793.171 were later 
computed, they proved strictly worse (for this maximize model). 


SAMPLE EXERCISE 12. 22: UNDERSTANDING PARENT BOUNDS 


Identify for node 5 in NASA branch and bound Figure 12.3 the best (upper) bound 
known for the optimal value of the corresponding candidate problem before the 
candidate’s LP relaxation was solved. 
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Analysis: Prior to solving the LP relaxation of the the candidate problem for node 
5, the best bound available on its optimal value comes from its parent. Parent bound 
pv = 791.250 is valid for child node 5 because the is identical except for the extra 
X9 = O constraint. 


Terminating with Parent Bounds 


One way a sophisticated branch and bound algorithm can exploit parent bounds 
12,39 | arises whenever a new incumbent solution is discovered. 


12.40 | Whenever a branch and bound search discovers a new incumbent solu- 
tion, any active partial solution with parent bound no better than the new 
incumbent solution value can immediately be terminated. 


Processing of partial solution 15 in Table 12.3 illustrates. An incumbent solution 
was discovered there with value ?) = 765. Earlier, partial solution 12 had been 
branched because its bound of ) = 757 was insufficient to terminate versus the 
incumbent at that time. Now we may terminate the still-active child with x5 = 1. The 
757 bound available from the parent is no better than the incumbent discovered at 
node 15. 


The following tree shows a minimizing branch and bound search that has processed 
4 partial solutions and just discovered its first incumbent solution at node 3. Nodes 
marked «a, 6, and y are active but not yet explored. 


by solving 


Determine which if any active partial solution can be terminated if the incumbent 
solution discovered at node 3 has value (a) } = 80; (b) } = 63. 


Analysis: We apply principle | 12.40], comparing with parent bounds 60 at node a 
and 69 at nodes f and y. 


(a) Incumbent value > = 80 is not as good as either parent bound for this minimize 
model. No active solution can be terminated. 


(b) Incumbent value > = 63 is not as good as 60, but superior to 69. Thus active 
partial solutions 6 and y should be terminated. 
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Stopping Early: Branch and Bound as a Heuristic 


Table 12.4 provides a historical summary of the NASA example in Figure 12.3 ane 
Table 12.3. Incumbent solutions reported in the first column show a typical patters 
Good incumbents were uncovered fairly early in the search, with the final 50 * 
80% of computational effort consumed in squeezing out the last improvements an¢ 
proving global optimality. For example, nearly optimal } = 755 was discovered afte 
only 7 of the 20 partial solutions analyzed. 


TABLE 12.4 Incumbent and Best Parent Bound History 
in NASA Example, Branch Bound Figure 12.3 


After Incumbent Best After Incumbent Best 

Node Value Parent Node Value Parent 
0 695 826.250 10 755 791.250 
1 695 826.250 tah (ES) 780.000 
2 695 826.250 12 755 780.000 
8 695 798.750 13 155 780.000 
4 695 798.750 14 TES) 770.500 
5) 725 798.750 15 765 766.909 
6 755 798.750 16 765 766.909 
7 755 798.750 17 765 766.909 
8 755 793.171 18 765 765.000 
9 755 793.171 


—— —_— -— 


Often, we are willing to settle for less that global optimality to avoid the lone 
final phase. That is, we want to stop early, accepting the last incumbent as a heuristic 
optimum. 


Bounds on the Error of Stopping with the Incumbent Solution 


Of course, we know that such incumbent solutions are feasible for the full problem 
and better than any other feasible solution that we have encountered. But we ca= 


use the best of current parent bounds | 12.39 | to conclude much more. 


value for 
Since every full solution that might still improve on the incumbent is a completio= 
of some active partial solution, we can bound every such solution by finding the bes 
of the corresponding parent bounds. 

The second column of Table 12.4 tracks this overall bound on remaining active 
partials in our NASA example. For example, at the moment after partial solutio= 
x© is explored and branched, the tree then contains active children of nodes 1, 3. 4 
5, and 6. The best of their relaxation bounds 


max{798.750, 793.171, 791.250, 787.000, 770.500} = 798.750 


is as much as any future incumbent could ever achieve. Any unexplored solution has 
to be a feasible completion in one of those active partial solutions. 
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Suppose now that we decide to terminate the search after node 6. Table 12.4 
shows that the incumbent solution value was 755 at that point, and we just com- 
puted the best parent bound of 798.750. Thus we can compute that our approximate 
optimum (the incumbent) is at most 


(best possible) — (best known) 798.750 — 755 


= 5.8% 


(best known) ie 755 


below optimal. That is, we can bound the error in our approximation. 


We use the lower bound in the denominator of percent computations because the 
el solution value could be that small. 


Reta to the minimizing ibratch and bound tree of Sample E Exercise 12. 23 he as- 
sume the incumbent solution value discovered at node 3 was ) = 71. Determine the 
maximum error and percent error in stopping the search at that point. 
Analysis: After node 3 the best parent bound | 12.41 | is 
min{60, 69} = 60 
_ Thus the maximum error for this minimize model is 
(best known) — (best possible) = 71 — 60 = 11 
Asa percent the maximum error is 


(best known) — (best possible) rs 71 —60 = 18.3% 
(best possible) Senet lcptarigiied is 


Depth First, Best First, and Depth Forward Best Back Sequences 


Another important implementation question in branch and bound is how to select 

a partial solution to pursue among the many that may be active. We have already 

introduced the simple depth first rule , which selects at each iteration an active 

partial solution with the most components fixed (1.e., one deepest in the search tree). 
Parent bounds | 42.39 | permit other alternatives. 


- ais : 


Best first search selects at each iteration an active partial solution with 


He branching a ae — one wit ic parent bound es termination. 
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All these rules require tie-breaking refinements when several partials have 
maximum depth or best parent bound. One alternative builds on fractional variable 


rule} 12.36 | by preferring the nearest child. 


12.45 When si sever sral activ tial solutions tie for dee pest or best parent bound, 
the nearest child rul e one with last fixed variable value nearest the 
corresponding component f the aes LP. relaxation. 


Assuming that the parent relaxation optimum tells us something about good values 
for the branching variable, this nearest child is more likely to lead to early discovery 
of good incumbent solutions. 

Figure 12.4 illustrates all 3 rules [12.28 |-[12.44] on NASA model (12.16). 
Branch and bound trees are shown after the first 10 partial solutions. Each uses 
nearest child tie-breaking rule |} 12.45}. 

Part (a) applies depth first search, always selecting an active partial solution 
with the most variables fixed. The deepest active partials in the current tree are the 
children of node 9. Nearest child rule | 12.45 would chose the one with x4 = 1 as 
x") because the parent LP relaxation had x4 = 0.6, which is closer to 1 than 0. 

Notice that depth first search will automatically choose a child of the last partial 
solution analyzed if that partial was branched. It was a deepest solution before 
branching, and the children have an additional variable fixed. This means that depth 
first search tends to move rapidly to fix enough variables that a feasible solution t 
the full model is uncovered. When rounding is not easy, that may be the best way te 
produce early incumbent solutions. 

Selecting a child of the last node investigated can also have important compu- 
tational savings. For example, at node 3 of Figure 12.4(a) the associated candidate 
problem differs by only the x14 = 0 constraint from that of parent node 1. Often, this 
similarity can be exploited to solve the relaxation at a child very quickly by starting 
at the LP optimum of the parent. 

Figure 12.4(b) illustrates best first search, always advancing to an active partial 
solution with best parent bound. At the moment depicted there are active childres 
of nodes 4, 7 and 9. We would next select a child of node 7 because its parent 
bound 


787 = max{780, 787, 757} 


is best for this maximizing model. Partial solution x“) would be the child of 7 with 
X= ile 

Here the idea is always to pursue a partial solution that could lead to the best 
possible completion. We select one with best parent bound because that is the mos: 
accurate information at hand about how good completions might be. 

Notice, however, that best first search tends to skip rapidly around the branch 
and bound tree, with selected partial solutions often rather different than the one 
just before. This tendency means that depth first’s efficiency when advancing to = 
child is often lost. 
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¥ = 826.250, ¥19 = 0.875, round = 695, ¥ = 695 


¥ = 793.171, %; = 0.78, 
V = 798.750, x44 = 0.75, 
round = 660 


round = 510 


V = 780, Xy = 0.60, 
round = 660 
by bound 


V = 791.250, Xo = 0.25, 
round = 540 


V = 714, Xq = 0.60, 
round = 660 


infeasible 


by bound 


(a) Depth first 


V = 798.750, X14 = 0.75, 
round = 660 


x44 = 0 


V = 793.171, ¥3 = 0.78, 
round = 510 


x3 =0 


V = 791.250, Xo = 0.25, 
round = 540 


V = 780, X% = 0.60, 
round = 660 


V = 713.5, X9 = 0.50, 
round = 538 


Xy = 0 


X= 1 


¥ = 787, ¥4 = 0.25, 


round = 725 
by bound v=725 by parent by parent 
y= 1 X= 0 after 7 


after 7 


V = 757, X4 = 0.40, 
round = 725 


x5=0 


(b) Best first 
»9£12.4 Alternative Partial Solution Selection in NASA Example 
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V = 826.250, ¥19 = 0.875, round = 695, V = 695 


7 = 793.171, % =O8 


F = 198.750, ¥44 = 0.75, 
é #4 ronndl= S10 


round = 660 
x14 = 0 


¥ = 780, ¥ = 0.60, 


V = 791.250, Xo = 0.25, 
round = 660 


round = 540 


Xy =0 


V = 787, X4 = 0. ra 
round = 725, V 


infeasible 


V = 770.5, X = 0.50, 
round = 755, V = 755 


V = 770.167 
by bound 
(c) Depth forward best back 


FIGURE 12.4 Alternative Partial Solution Selection in NASA Example (Continued) 


The depth forward best back rule of Figure 12.4(c) (and full search tree, Figure 
12.3) provides a compromise. As long as nodes are branched, so that their childres 
are a possible next choice, this rule follows depth first in selecting one of the childrer 
When a node is terminated, however, so that some disruption is unavoidable, rulc 
12.44 | pursues the more ambitious best-first policy. 

In Figure 12.4(c), partial solution x was terminated. Thus x“® would be a= 
active partial with best parent bound. Here that would be the child of node 3 wit® 
x3 = 0. Had node 9 been branched, x“ would have been one of its children. 


SAMPLE EXERCISE 12.25: s ING AMONG ACTIVE PARTIAL SOLUTIONS 


Return to the maximizing Benen and bound problem of Sample Exercise 12.19 anc 
show the branch and bounds trees for searches guided by (a) depth first selection 
(b) best first selection; (c) depth forward best back selection. Use nearest chile 


tie-breaking rule | 12.45 | in each case. 


Analysis: 
(a) The branch and bound tree for depth first rule | 12.28 | is 
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x = (0.2, 1,0, 0), ¥ = 82.8 


by bound 
by bound 


x9 = (0,1, 1, 0.5), 
v=77 


by solving infeasible 
v=T71 


| (b) The branch and bound tree for best first rule | 12.43 | is 


€ = (0.2, 1, 0, 0), V = 82.8 


x@) = (1,0,0,0), ¥=74 x® = (0, 1, 0.67, 0), 7 = 80.67 


by solving 
¢=74 xg=1 


x = (0,1, 0,2), 7 = 28 
by bound 


x) = (0,1, 1, 0.5), 

v=T77 

by solving infeasible 
%=77 


c) The branch and bound tree for depth forward best back rule |_ 
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x = (0.2, 1,0, 0), 7 = 828 


x = (1,0,0,0), 7=74 x = (0, 1, 0.67, 0), ¥ = 80.67 


by bound 
x3 = 1 


by bound 


v=T717 
by solving infeasible 
Va77 


Branch and Cut Search 


Section 12.3 showed that relaxations can often be strengthened by adding new, valid 
inequality constraints. A variation of branch and bound termed branch and cut takes 
advantage of such inequalities to speed computation. 


all feasible solutions to the full discrete ade. but they should cut off (render 
infeasible) the last relaxation optimum. 


Branch and Cut Solution of the River Power Example 


Algorithm 12B provides a formal statement of the branch and cut process. To see 
the idea, return to our River Power example of Section 12.4. Decisions there relate 
to which generators to activate, and the model is 


min 7x; + 12x. + 5x3 + 14x4 (total cost) 
st. 300x; + 600x2 + 500x3 + 1600x4 > 700 (demand) (12.17 
X1,X2,.%3,%4 =Oorl 
Solution of the first, all-free candidate problem produces relaxation optimum 
«© — (0,0, 0,0.438), t= 6.125 


In normal branch and bound Figure 12.2 we immediately branched on fractiona! 
variable x4. 
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Step 0: Initialization. Make the only active partial solution the one with all 
discrete variables free, and initialize solution index  < 0. If any feasible 
solutions are known for the model, also choose the best as incumbent solution 
% with objective value }. Otherwise, set ) < —oo if the model maximizes 
and { < +00 if it minimizes. 

Step 1: Stopping. If active partial solutions remain, select one as x®, and 
proceed to Step 2. Otherwise, stop. If there is an incumbent solution &, it is 
optimal, and if not, the model is infeasible. 

Step 2: Relaxation. Attempt to solve the linear programming relaxation of 
the candidate problem corresponding to x. 

Step 3: Termination by Infeasibility. If the LP relaxation proved infeasible, 
there are no feasible completions of partial solution x. Terminate x®, 
increment ¢ < ¢+ 1, and return to Step 1. 

Step 4: Termination by Bound. If the model maximizes and LP relaxation 
optimal value i satisfies  < }, or it minimizes and i > 5, the best feasible 
completion of partial solution x cannot improve on the incumbent. Terminate 
x, increment ¢ <— t+ 1, and return to Step 1. 

Step 5: Termination by Solving. If the LP relaxation optimum x satisfies 
all binary constraints of the model, it provides the best feasible completion 
of partial solution x. After saving it as new incumbent solution by & — & 
and j < 3, terminate x, increment t — t+ 1, and return to Step 1. 

Step 6: Valid Inequality. Attempt to identify a valid inequality for the full 
ILP model that is violated by the current relaxation optimum x. If successful, 
make the constraint a part of the full model increment t — ¢ +1, and return 
to Step 2. 

Step 7: Branching. Choose some free binary-restricted component x, that 
was fractional in the last LP relaxation optimum, and branch x by creating 
two new actives. One is identical to x except that x, is fixed = 0, and the 
other the same except that x, is fixed = 1. Then increment t — ¢+1 and 
return to Step 1. 


Before turning the current partial solution into two in that way, branch and cut 


Algorithm 12B would try to improve the relaxation. The idea is to find an inequality 
satisfied by every binary solution to the full model but violated by ¥. 


Methods used to find such cutting inequalities vary enormously from one model 


to another. In this example we simply observe that any feasible solution in (12.17) 
must have a least one generator turned on. Thus 


Xytx. +43 4+%4>1 (12.18) 


is valid. Also, constraint (12.18) cuts off previous relaxation optimum *© because 


0+0+0+0.438 # 1 
Figure 12.5 shows that branch and cut advances by adding inequality (12.18) 


to improve the relaxation. With the same all-free partial solution, we now obtain 
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stronger results: 
O'S O; OVCBISHONS2)p""™ 1 161636 
Suppose now that a hunt for further cuts meets with no success. The searc® 
branches as usual on fractional variable x3. Depth first rule} 12,28 | with nearest chile 


tie-breaker | 12.45 | makes the next partial solution x™ = (#, #, 1, #). 


At node 2 the relaxation again proves inadequate for termination. This time 
analysis of possible cuts discovers violated inequality 


Xy +x. +43 +2x4 > 2 (12.18 


which recognizes that either generator 4 or two others are required to meet the 70- 
megawatt output requirement. The improved relaxation at node 3 has ) = 12, anc 
the search continues. 

Notice that both inequalities (12.18) and (12.19) are valid for the original mode 
(12.17). That is, they do not depend on variables fixed in partial solutions. As = 
consequence, they may be retained when candidate problem relaxations are solvec 


at subsequent nodes 4 to 6. Each new cut discovered strengthens all subsequen® 
relaxations. 


(py x) = (0, 0, 0, 0.438) 
Se V = 6.125 


> 
I 
8 


Xy+X_4+%3+%X,21 


x = (0, 0, 0.818, 0.182) 
7 = 6.636 


x@ = (0, 0,1, 0.125) 


x = (1, 0, 0, 0.5) 
V = 6.750 


v=14 


by bound 


Xy +X 4+X3+2x422 


x) = (0, 0, 1, 0.500) 
¥ = 12.000 


x4 = (0,0, 1,1) 
v=19 


by solving by solving 
v=19 %=12 


¥©) = (1,0, 1,0) 
v=12 


Figure 12.5 Branch and Cut Search of River Power Example 
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The following is isa pemicay mel cut tree een Tea CoTAT of msc 12B toa 
maak: aed: linear program over x1, X2,x3 = 0 or 1 and x4 > 0. The best first 


Gs co fey ¥ = (0.5, 0, 0,2) 


Vv = 63 


xX, +%321 


nO = aCe 0.8, 0.7, 0) 
v= 


2x1 +X 22 


2 = (0.6, 0.8, 0.4, 1) 
v= 53 


¥9) = (0.5, 1, 0.6, 0.1) 
Vv = 50 


¥©) = (1,0, 1, 6.2) 
v= 45 


by solving 
X3a+X4 rp) v= = 45 


x) = (0.5, 1, 0.7, 4.3) 
Vv = 48 


¥ = (0.5, 1,1, 4) 
v= 42 


/ by bound infeasible 


Assuming that all cutting inequalities shown are valid, trace the computation and 
justify each step. 


Analysis: There is no initial incumbent solution. The first relaxation at node 0 proves 
feasible and fractional, but cut x; +3 > 1 could be generated. We can usefully add 
it because 


xO 4 -0540=05¥1 


The improved relaxation at node 1 still cannot be terminated, but valid inequality 
2x; + x2 > 2 can be added because 


2x? + x = 20.3) + (0.8) = 14 #2 


680 Chapter Twelve e Discrete Optimization Mcthods 


The resulting node 2 relaxation remains feasible and fractional, and no further cuts 
are added. Now we branch on fractional variable x2, choosing the child with x. = 1 


as partial solution 3 under the nearest child rule . Its LP relaxation also fails 
to justify termination, but another valid inequality x3 + x4 => 5 is added with 
x9) 4+. =06401=0.7 75 

After branching on fractional variable x3, there are active children of nodes 
2 and 4. Best first enumeration rule takes us next to partial solution x® = 
(#, 0, #, #) because parent bound 

53 = max{53, 48} 
The resulting relaxation optimum is feasible in the original model and so optimal im 
the corresponding candidate problem. It provides incumbent solution 
X= (50, £62); d= 45 
Now returning to the children of node 4, the search goes first to the child wit® 


x3 = 1(rule} 12.45]). Relaxation bound b = 42 is now sufficient to terminate becaus: 
42 < 45. The search finishes by discovering that the relaxation of node 7 is infeasible 
Final incumbent solution xX is optimal. 


12.6 IMPROVING SEARCH HEURISTICS FOR DISCRETE 
OPTIMIZATION INLPS 


Many large combinatorial optimization models, especially INLPs with nonlinear o>- 
jective functions, are too large for enumeration and lack strong relaxations that are 
tractable. Still, much can be done. Suitable adaptations of improving search methods 
introduced in Chapter 3 can often yield very effective heuristic algorithms. That 
we can still find good feasible solutions even though we will not be able to guarantes 
their optimality or even be sure about how close they come to optimal. 


Rudimentary Improving Search Algorithm 


Algorithm 12C shows a rudimentary adaptation of improving search to discrete moc- 
els. Like the continuous cases of Chapter 3, the process begins with an initial feasib’< 
solution x, Each iteration ¢ considers neighbors of current solution x and tries * 
advance to one that is feasible and superior in objective value. Ifno feasible neighbo 
is improving, the process stops with local optimum and heuristic optimum x”. 


Discrete Neighborhoods and Move Sets 


What is new about the discrete form of improving search is that we must explicit 
define the neighborhood of a current solution. Unlike the continuous case, where 
there are infinitely many points near a current solution, discrete search must advance: 
to a binary or integer point. Explicit move sets (denoted M) control what solution 
are considered neighbors of current x. 


12.47 | Improving searches over discrete variables define neighborhoods by spec- 
_ ifying a move sct M of moves allowed. The current solution and all reachable 
- from it in a single move Ax € M comprise its neighborhood. 
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Step 0: Initialization. Choose any starting feasible solution x, and set 
solution index t < 0. 

Step 1: Local Optimum. If no move Ax in move set M is both improving 
and feasible at current solution x®, stop. Point x is a local optimum. 

Step 2: Move. Choose some improving feasible move Ax « M as Ax +), 


Step 3: Step. Update 
xt) - yO 1 Ax) 


Step 4: Increment. Increment ¢ — f+ 1, and return to Step 1. 


Consider the discrete optimization model 
max 20x; — 4x. + 14x3 
s.t. 2x, + x2 + 4x3 <5 
Ki, Xo a, = Olor! 
and assume that an improving search begins at x = (1, 1, 0). 


(a) List all neighbors of x under move set 


“WOOO 


(b) Determine which members of the neighborhood are both improving and feasible. 


Analysis: 


(a) Following definition [12.47], the neighbors of x under the specified move set 
are 

(1, 1,0)+(,0,0) = (,1,0) 

(1, 1, 0) + (—1, 0, 0) = (0,1, 0) 

(1,1,0)+0,1,0) = (,2,0) 

(1, 1,0) + (0, -1,0) = (4, 0,0) 

1k. 0) +0; Os) eal I, 1) 

(1, 1, 0) + (0,0, -1) = (1,1, -1) 


b) Of the neighbors in part (a), only x = (0, 1,0), which has objective value —4, 

and x = (1, 0, 0), which has objective value 20, are feasible for all constraints of the 
model. Current point x = (1, 1, 0) has objective value 16. Thus x = (1, 0, 0) is the 
only neighbor that is both improving and feasible (i.e., the one to which improving 
search would advance), 
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NCB Example Revisited 


We will use the NCB example of Section 11.5 to illustrate improving search in discrete 
optimization. Recall that we seek a shortest-distance routing through 10 points = 
a printed circuit board that must be drilled. Table 12.5 details hole-to-hole trave 
distances. 

For improving search it will be most convenient to employ the quadratic as- 


signment formulation | 11.27] of Section 11.5: 


10 
min = > 3 dij VRE, _ (total distance) 


ee 


= Yei=l forall k= 1,224,410 (some hole each k) (12.2 


va Ver = foralla=1,.:., 10 (each i assigned) 


k=1 
Yei = Oor 1 for allk =1,..., 10; i=1,...,10 
where 
leek if kth hole drilled is i 
Jkt = 0 otherwise 


and yi9+1, is understood to mean y,,; in objective function summations. 

We will abuse notation in the usual way to think of solutions as vectors y eves 
though components have two subscripts. The NCB optimal solution y*, which = 
depicted in Figure 12.6, has length 81.8 inches and nonzero components 

Via =Y23 = V3.6 = Ya10 = Ys9 = Y65 = V7.8 = 3,7 = V9.4 = Yio2 = 1 
We begin our improving searches with initial feasible solution 
yO =. 05) 250305 1)052.4, OF 25-0)2. 2051) (122 
corresponding to yi; = y22 = +--+ = yio,10 = 1. That is, hole 1 is drilled first, thes 
hole 2, and so on. The total length is 
dy,2 + do3 + d34 + das + ds6 + do,7 + dg + dg9 + do,10 + dio,1 
= 3.6+3.64+714+7.0+ 9.9 + 25.0 + 10.3 + 10.2 + 3.6 + 26.4 
= 106.7 inches 


TABLE 12.5 Distances between Holes in NCB Example 


1 —, 3.6. 5.1) 1010. 15:3. 2010, 16.0), 14-2. 23:0|, 26.4 
z) 3.6 —. 3:6— 36:4= 12 18d 13, 20:6 19.7 = 231) 
3 Sl 3.6 e106 1510 Se LOS ea ean 
Abt 010 fesG:4s veal Sea OelSy/ al00b 24 25en3:o™ M0 
peasy s) eel aI soy rae e7/0) = 9:9 SIS Oe Smiles 
6 2010 sa :0 sala 99 —— 925.0 lA Ie. 150) 
(e216 OFT 25s:8 st 1008F tse 250 — 0S ASAE 
Se 142 sD Orel Se A De S04 OOS — 102 13.0 
D> 23.0 yl One 18:4 SO Opel 20 eNO Zee Ore, — 3.6 
dO eae bint) rey Le S170) oo i ye SO LO EO we B0) — 
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i 10_| the NCB Example 
Choosing a Move Set 


The critical element of a discrete improving search heuristic is its move set. If it were 
possible, we would make every solution a neighbor of every other. Then the search 
would yield global optima because a stop implies that no solution at all is feasible 
and superior in objective value to the current. 

In a practical search, however, we must accept much less. 


€ set M ofa discrete i improving search must be compact enough 
e at each iteration for i improving feasible neighbors. 


On the other hand, we would not want too limited a move set. 


12.49 i The solution produced. by ¢ a discrete i improving search depends on the 
et (or neighborhood) emplayee= = piers moye sets eeneEly testi 
: superior local optima. 


If the move set M of a discrete improving search is too restrictive, very few solutions 
will be considered at each iteration, and poor-quality local optima will result. 

We will adopt one of the simplest move sets for our NCB example. Specifically, 
our M will consist of pairwise interchanges swapping one position k with another &. 
Corresponding move vectors Ay have two —1 components at deleted assignments, 
two +1 components at revised ones, and all other components 0. For example, if the 
hole in route position k = 3 is now number 7, and the hole in position € = 5 in now 
number 1, the corresponding interchange move direction has 


Ays7=—1,  Aysi=—l, Aysi=+1, © Ays7=+1 
changing hole 1 to position 3 and hole 7 to position 5. 

In all, this pairwise interchange M contains a Ay for (10 - 9)/2 = 45 choices 
of k and &, each with 10-9 = 90 possible current hole assignment pairs—a total 
of 45 - 90 = 4050 moves. However, at any particular solution y, only 45 moves 
mterchanging its specific assignments lead to a feasible neighbor. In all 


searches in this section we adopt the one such move most improving the objective 
function. 
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| SAMPLE. EXERCISE: 
Return to the discrete model of Sails Exercise 12. 27 at initial point xO = = (ee 
(a) Show that x© is not locally optimal under the move set of Sample Exercise 12 


(b) Show that x© is locally optimal over smaller move set 
M & {(1, 0, 0), (0, 1,0), 0, 0, 1)} 


Analysis: 


(a) Sample Exercise 12.27(b) established that x = (1, 0, 0) is a feasible neighbor 
x with superior objective value. Thus x is not best in its neighborhood, and s 
not locally optimal. Algorithm 12C would advance to x = (1,0, 0) and repeat. 


(b) Over this more restricted move set, neighbors are 


(1,1,0) + (4, 0,0) = (2,1, 0) 
(1, 1,0) + (0, 1,0) = (1,2, 0) 
(,1,0) + (0,0,1) = (1,1, 1) 


None is feasible, so x is locally optimal. 


Rudimentary Improving Search of the NCB Example 


For initial solution (12.21) in our NCB example, the objective function impact of 
45 feasible pairwise interchanges is as follows: 


Nee 2 3 4 5 6 7 8 9 10 
ie S19) TSG OSG 0 ot ule mos Oslee o8l8 
2 0.8 115 263 182 180 302 54.0 344 
3 64 136 146 30 178 418 22.8 
4 93° 274 - 16 95.14 19:5 — 13.0 
5 =f; 22 eas td 52 8D 
6 10.0 -31 82 -0.6 
7 = lulls aadideo2 dt 
8 18.3 -1.5 
9 ~0.6 


For example, the best swap, position k = 4 for £ = 6, implies savings of 
d34 + dss +ds6+d67 =714+7.0+ 9.9 + 25.0 = 49 inches 
for delinking holes 4 and 6 from their current fourth and sixth tour positions, p= 
costs 
d36 +d65+ds4+d47 = 15.0+9.9+7.0 + 10.0 = 41.9 inches 


for relinking in their new positions. The net change is 41.9 — 49 = —7.1 inches. 

Table 12.6 displays the tour sequence resulting from this best interchange. 
also details, swaps, move directions, and solutions visited to reach local optimality « 
t = 5. Local optimum y has length 92.8 inches with 


¥1.2 = 921 = 93,3 = ¥46 = 95,5 = Vo4 = 7,8 = Ya,9 = Yo,10 = J10,7 = 1 
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TABLE 12.6 Rudimentary Improving Search of the NCB Example 


t Drill Sequence Length Interchange Nonzero Move Components 

Q  1-2-3-4-5-6-7-8-9-10 106.7 4th for 6th Ay44=Ayeo=-1, Ayag=Ayoa4=1 

1 1-2-3-6-5—-4—7-8-9-10 99.6 Ist for 2nd Ay,,1=Ay22=-l, Ayj.= Ayo1=1 

2  2-1-3-6-5-4-7-8-9-10 O77, 8th for 10th Ayg g=AYto,10=-1, Ays 10= AYj0,8= 1 
3 2-1-3-6-5—4-7-10-9-8 96.0 7th for 10th Ay7,7=AyYio,g=—L, Ay7,g=Ayio,.7=1 
4 2-1-3-6-5—4-8-10-9-7 93.8 8th for 9th Ayg 10=Ayo,9=—1, Ayg y=Ayg 19 =1 

5 2-1-3-6-5—4-8-9-10-7 92.8 Local optimum 


Initial length 106.7 inches has been reduced by 13%, but locally optimal value 92.8 
leaves us well above globally shortest tour length 81.8 inches. 


SAMPLE EXERCISE 12.29: PERFORMING DISCRETE IMPROVING SEARCH 
Consider the knapsack model (Section 11.2) 


max 18x; + 25x2 + 11x3 + 14x4 
s.t. 2x1 + 2x. +x3 4x4 <3 
Kil,» ena Ororil 


under the single complement move set admitting moves that change one 0-component 
to 1 or one 1-component to 0. Compute an approximate optimal solution by discrete 
improving search Algorithm 12C beginning at x = (1,0,0,0). If more than one 
neighbor is feasible and improving at any move, choose the one that improves the 
objective the most. 


Analysis: Feasible neighbors of the givenx are (0, 0, 0, 0), (1, 0, 1, 0) and (1, 0, 0, 1) 
with objective values 0, 29, and 32, respectively. The last improves the objective 
function the most, so the search advances to x = (1,0, 0,1). 

Atx, feasible neighbors are (0, 0, 0, 1) and (1, 0, 0, 0). Since neither improves 
the objective function, the search stops with local optimum & = x = (1,0, 0,1) at 
objective value ) = 32. 


Multistart Search 


For our tiny NCB example we were able to know how far heuristic optimum sequence 
2-1-3-6-5—4-8-9-10-7 is from globally optimal. In a larger example it would be 
difficult to tell. Thus it is natural to at least try for further improvement. 

One obvious approach is multistart—repeating the search from several differ- 
ent initial solutions y. 


from different starting solutions is one way to improve the heuristic solutions 
produced byimproving search, > : 


Table 12.7 details searches of the NCB example from 3 different starts. The 
frst is the search of Table 12.6 with local minimum 92.8. Search 2 yields an improved 
local minimum with length 84.7 inches after only one iteration. Search 3 terminates 
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TABLE 12.7 Multistart Search of the NCB Example 


t Drill Sequence Length Interchange Nonzero Move Components 

Search 1 
0  1-2-3-4-5-6-7-8-9-10' 106.7 4th for 6th Ayaa=Ayo,6=-1, Ayva6=Avo,4= 1 
1 1-2-3-6-5-4-7-8-9-10 99.6 1st for 2nd Ay,,1=Ay2.2=-1, Ayy.=Ayo1=1 
2  2-1-3-6-S—4-7-8-9-10 97.7 8th for 10th Ayg.g=Ayio,i0=—1, Avs 10 = Avi0.s = 
3 2-1-3-6-5—4-7-10-9-8 96.0 7th for 10th Ay7,7=Ayio,g=—1, Ay7.3=Ayi0,7 = 
4  2-1-3-6-5—4-8-10-9-7 93.8 8th for 9th Ayg 19 = A¥o,9=-1, Ayg 9 =AYo,10 = + 
5 2-1-3-6-5—4-8-9-10-7 92.8 Local optimum Incumbent value = 92.8 

Search 2 
0  1-2-7-3-4-8-5-6-9-10 100.8 1st for 3rd Ay,,1=Ay3,7=-1, Ayi,7=Ay31=1 
1 7-2-1-3-4-8-5-6-9-10 84.7 Local optimum Incumbent value = 84.7 

Search 3 
0 1-10-2-9-3-8-4-7-5-6 157.7 1st for 4th Ayy,1=Ayag=-l, Ayy9=Aygi1=1 
1  9-10-2-1-3-8-4-7-5-6 97.5 6th for 8th Aye.g=Ayg.7=-1, Ayo7=Aygg=! 
2 9-10-2-1-3-7-4-8-5-6 92.2 3rd for 6th Ay3.2=Ayo,7=—l, Ay3.7=Ayo2=! 
3. 9-10-7-1-3-2-4-8-5-6 86.8 Sth for 6th Ays.3=Aye,.=-1, Ays.2=Aye3= 1 
4  9-10-7-1-2-3-4-8-5-6 86.0 4th for Sth Ay41=Ays2=—1, Aya. =Ays,=1 
5 9-10-7-2-1-3-4-8-5-6 84.7. Localoptimum Incumbent value = 84.7 


with the same solution after a longer sequence. 
these as an approximate optimum. 


Multistart would report the best « 


SAMPLE EXERCISE 12.30: PERFORMING MULTISTART SEARCH 
Return to the knapsack problem of Sample Exercise 12.29: 


max 18x; + 25x2 + 11x3 + 14x4 
st. 2x, + 2x2 +43 +24 <3 
Mist va Oor | 


and its single complement search neighborhood. Perform a multistart discrete im 
proving search starting from initial solutions (1, 0, 0, 0), (0, 1, 0, 0), and (0, 0, 1, 0 


Analysis: Sample Exercise 12.29 already established that x = (1,0,0,0) lead 
to local maximum x = (1, 0,0, 1) with value >) = 32. Now restarting with x“ = 
(0, 1, 0, 0), the best feasible neighbor is x = (0, 1, 0, 1) with value 39. 

This solution is locally optimal because no neighbor is both feasible and im 
proving. Since it also improves on the current best (or incumbent) solution, ou 
approximate optimum becomes x = (0,1, 0, 1), b = 39. 

The third search starts with x = (0,0,1,0). Its feasible neighbor that mos 
improves the objective function is x” = (0, 1, 1, 0) with value 36. Again, Algorith= 
12C stops at a local maximum. However, this one is not better than incumbent vale: 
D = 39, so it is not retained. 
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12,7 TABU, SIMULATED ANNEALING, AND GENETIC 
ALGORITHM EXTENSIONS OF IMPROVING SEARCH 


Although improving search Algorithm 12C can be quite effective on many models, 
expecially if applied repeatedly as in multistart, the improving search idea can be 
extended in a variety of ways to produce more robust algorithms. In this section we 
briefly introduce three of the most popular: tabu search, simulated annealing, and 
genetic algorithms. 


Difficulty with Allowing Nonimproving Moves 


An alternative to restarting improving search when no improving feasible move 
remains is to escape a local optimum by allowing nonimproving feasible moves. A 
few such retrograde moves might very well take the search to a region where progress 
can resume. 

Unfortunately, this appealing strategy has a fatal flaw (unless new technology is 
introduced). Consider, for example, the locally optimal point where the first search of 
Table 12.7 terminated. No improving move was available at final drilling sequence 2— 
1-3-6-5S—4—8-9-10-7 with value 92.8 inches. However, the best nonimproving move, 
which swaps the 9 in the eighth position for the 10 in the ninth, increases the objective 
by only 1.0 inch. 

Why not take it and hope it leads to a better local optimum? Try. Sequence 2-1- 
3-6-5—4-8-10-9-7 results with length 93.8 inches. There is now a feasible interchange 
that improves (perhaps only one): swap back the 10 and 9 in the eighth and ninth 
tour positions to reduce length to 92.8 inches. Adopting that move returns us to 
exactly where we began, and the search will cycle forever. 


Tabu Search 


Several schemes for incorporating nonimproving moves in improving search without 
undo cycling have proved effective on a variety of discrete optimization problems. 
One is called tabu search because it proceeds by classifying some moves “tabu” or 
forbidden. To be more specific, 


The effect is to prevent short-term cycling, although solutions can repeat over a 
longer period. 

Algorithm 12D gives a formal statement of this tabu modification to improv- 
ing search Algorithm 12C. A tabu list records forbidden moves, and each iteration 
chooses anon-tabu feasible move. After each step, a collection of moves that includes 
any returning immediately to the previous point is added to the tabu list. No such 
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Step 0: Initialization. Choose any starting feasible solution x and an 
iteration limit tmax. Then set incumbent solution & <— x and solution index 
t <0. No moves are tabu. 

Step 1: Stopping. If no non-abu move Ax in move set M leads to a feasible 
neighbor of current solution x, or if t = tmax, then stop. Incumbent solution 
x is an approximate optimum. 

Step 2: Move. Choose some non-tabu feasible move Ax « M as Ax? 

Step 3: Step. Update 


xt) ae) + Ax) 


Step 4: Incumbent Solution. If the objective function value of xt” is 
superior to that of incumbent solution X, replace X — x). 

Step 5: Tabu List. Remove from the list of tabu of forbidden moves any tha? 
have been on it for a sufficient number of iterations, and add a collection of 
moves that includes any returning immediately from xt to x, 

Step 6: Increment. Increment ¢ <— t+ 1, and return to Step 1. 


move is allowed for a few iterations, but eventually all are removed from the tabs 
list and again available. 

Since steps may either improve or degrade the objective function value. = 
incumbent solution X tracks the best feasible point found so far. When the sears 
stops, which is usually when user-supplied iteration limit fmax is reached, incumber 
X is reported as an approximate optimum. 


Tabu Search of the NCB Example 


Table 12.8 illustrates an implementation of tabu search on the NCB example. Figure 
12.7 tracks the objective function values of points encountered. 

The initial solution and move set of this tabu search are identical to those of the 
ordinary improving search in Table 12.6. This time, however, tmax = 50 points were 
visited, with search proceeding to the feasible, non-tabu neighbor with best objective 
function value, whether or not it improves. 

The design of tabu searches requires some judgment in deciding what moves 
to make tabu at each iteration. Too few will lead to cycling; too many inordinate 
restricts the search. 

Table 12.8’s search of the NCB example fixed the first of each two position 
interchanged for a period of 6 iterations. For example, after the first interchange 
fourth and sixth positions, no move again changing the fourth position was allows. 
for 6 steps. Tabu positions are underlined. Such a policy maintains a relatively n& 
set of available moves, yet prevents immediate reverses. 

Figure 12.7 shows clearly how tabu search improves as long as improving move 
are available, then begins controlled wandering. In the beginning the incumbent s> 
lution y improves rapidly. (Incumbent values are recorded in Table 12.8.) Late 
progress slows. Still, the global optimum was discovered on iteration t = 44. Nat 
urally, it would be the final incumbent reported when iteration limit tnax = 50 wa 
reached. 
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TABLE 12.8 Tabu Search of NCB Example 


Drill Sequence* Length Incumbent Interchange A Obj. 
1-2-3-4-5-6-7-8-9-10 106.7 106.7 4th for 6th Wal 
1-2-3-6-5—4-7-8-9-10. 99.6 99.6 1st for 2nd 19 
2-1-3-6-5—4-7-8-9-10 97.7 97.7 8th for 10th ily 
2-1-3-6-5—4-7-10-9-8 96.0 96.0 7th for 10th 2.2 
2-1-3-6-5—4-8-10-9-7 93.8 93.8 6th for 10th 2.3 
2-1-3-6-5-7-8-10-9-4 96.1 93.8 Sth for 10th ail 
2-1-3-6-4—7-8-10-9-5 96.2 93.8 4th for 10th 2.9 
2-1-3-5—4-7-8-10-9-6 93.3 93:3 1st for 3rd 1.6 
3-1-2-5—4-7-8-10-9-6 91.7 91.7 8th for 9th —0.2 
3-1-2-5—4-7-8-9-10-6 91.9 91.7 2nd for 3rd -1.7 
3-2-1-5-4-7-8-9-10-6 93.6 91.7 6th for 7th 3.2 
3-2-1-5—4-8-7-9-10-6 96.8 91.7 Sth for 7th -3.0 
3-2-1-5—7-8-4-9-10-6 99.8 91.7 4th for 7th 15.9 
3.2-1-4-7-8-5-9-10-6 83.9 83.9 1st for 3rd Sor 
1-2-3-4-7-8-5-9-10-6 86.0 83.9 8th for 9th -0.5 
1-2-3-4-7-8-5-10-9-6 86.5 83.9 2nd for 3rd -0.8 
1-27-85-9-10463 963 83.9 Istfor2nd = -1.3 
2-1-7-8-5—9-10—4-6-3 97.6 83.9 8th for 9th 9.9 
2-1-7-8-5-9-10-6-4-3 87.7 83.9 2nd for 9th 0.9 
2-4-7-8-5-9-10-6-1-3 86.8 83.9 9th for 10th 5.0 
2-4-7-8-5-9-10-6-3-1 81.8 81.8 6th for 7th -0.5 
2-4-7-8-5-10-9-6-3-1 82.3 81.8 5th for 7th -3.1 
2-4-7-8-9-10-5-6-3-1 85.4 81.8 1st for 10th 2.1 
1-4-7-8-9-10-5-6-3-2 87.5 81.8 7th for 8th 0.7 
1-4-7-8-9-10-6-5-3-2 86.8 81.8 2nd for 3rd 0.1 
1-7-4-8-9-10-6-5-3-2 86.7 81.8 9th for 10th -3.0 
1-7-4-8-9-10-6-5-2-3 89.7 81.8 Stop 

“ Underlining indicates components not allowed to change. 

105 in. 
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FIGURE 12.7 Solution Values in Tabu Search of NCB 
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Results will vary with models and details of tabu policy, but there is good reason 
to believe that such performance is typical. Suitable implementations of the tabu 
variation on improving search can greatly enhance the quality of heuristic solutions 
obtained. 


Return to the knapsack problem 


max 18x; + 25x. + 11x3 + 14%4 
s.t. 2x1 + 2x. +43 4+%4 <3 
Moy on og ka = Oo 


of Sample Exercises 12.29 and 12.30 and assume that we will use the same single- 
complement move set. 


(a) Explain how making it tabu to complement any component x; for the next = 
iterations after it is changed by the search prevents short-term cycling. 


(b) Begin from solution x = (1,0, 0, 0) and use this tabu rule in executing Algo- 
rithm 12D through tmax = 5 steps. 


Analysis: 


(a) Once a component is changed for 0 to 1, or vice versa, the only way to return tc 
the immediately preceding solution is to complement the same component again. 


(b) The required search is summarized in the following table: 


t x Value Incumbent Value Complemented A obj. 
0 (1,0, 0,0) 18 18 j=4 14 
1 (1,:0,0)0) 32 32 jae -18 
2 (0,0,0, 1) 14 By) =e 25) 
3 (Oi 10,1) 39 39 j=4 -14 
4 (0,1,0, 0) 25 39 j=3 11 
5 (0,1,1,0) 36 39 Stop 


Each iteration begins by selecting an x; to complement that is not among those 
marked tabu (underlined) and does preserve feasibility. The best such move produces 
the next solution. The result is also saved as a new incumbent if it is superior to any 
feasible solution encountered so far. Computation stops with approximate (here 
exact) optimum x = (0,1, 0, 1) att = thax = 5. 


Simulated Annealing Search 


Another method of introducing nonimproving moves into improving search is terme 
simulated annealing because of its analogy to the annealing process of slowly cooling 
metals to improve strength. 


12.53 | Simulated annealing : 
ing moves according to probabili 
numbers. 


Improving and accepted nonimproving moves are pursued; rejected ones are not. 


tt 
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Algorithm 12E provides details. The move selection process at each itera- 
tion begins with random choice of a provisional feasible move, totally ignoring its 
objective function impact. Next, the net objective function improvement Aobj (non- 
positive for nonimproving moves) is computed for the chosen move. The move is 
always accepted if it improves (A obj > 0), and otherwise 


probability of acceptance = e4°i/7 (1232) 


That is, all improving moves and some nonimproving ones are accepted. The proba- 
bility of accepting a nonimproving move declines as net objective improvement Aobj 
becomes more negative. 


SIMULATED ANNEALING SEARCH = 


Step 0: Initialization. Choose any starting feasible solution x, an iteration 
limit tmax, and a relatively large initial temperature g > 0. Then set incumbent 
solution X — x and solution index ¢ <— 0. 

Step 1: Stopping. If no move Ax in move set M leads to a feasible neighbor 
of current solution x, or if f = tmax, then stop. Incumbent solution & is an 
approximate optimum. 

Step 2: Provisional Move. Randomly choose a feasible move Ax « M 
as a provisional Ax“*”, and compute the (possibly negative) net objective 


function improvement Aobj for moving from x to (an + Ax‘) (increase 


for a maximize, decrease for a minimize). . 
Step 3: Acceptance. If Ax“ improves, or with probability ¢4°i/9 if Aobj 
< 0, accept Ax“*” and update 


xD - xO 4 Ayer) 


Otherwise, return to Step 2. 

Step 4: Incumbent Solution. If the objective function value of xt) is 
superior to that of incumbent solution x, replace ¥ <— x“). 

Step 5: Temperature Reduction. |f a sufficient number of iterations have 
passed since the last temperature change, reduce temperature q. 

Step 6: Increment. Increment t — ¢+1, and return to Step 1. 


Parameter qg in (12.22) is a temperature controlling the randomness of the 
search. If q is large, the exponent in (12.22) approaches 0, implying that the prob- 


ability of accepting nonimproving moves approximates e? = 1. If q is small, the 


probability of accepting very bad moves decreases dramatically. Simulated an- 
nealing searches usually begin with g relatively large and decrease it every few 
iterations. 

As with tabu and other searches that can make nonimproving moves, an in- 
cumbent solution X must be maintained to keep track of the best feasible solution 
found so far. When computation stops, x is output as an approximately optimal 
solution. 
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Simulated Annealing Search of NCB Example 


Table 12.9 provides an abridged summary of a simulated annealing search of ow 
NCB example. Figure 12.8 shows the complete history of accepted solutions throus 
iteration limit tax = 50. Like all other searches of this section, move set M includs- 
all single interchanges, and initial solution y is the one in (12.21). 


TABLE 12.9 Simulated Annealing Search of NCB Example 


t Drill Sequence Length Incumbent Tempq _ Interchange AObj. Outcome 


QO 1-2-3-4-5-6-7-8-9-10 106.7 106.7 5.00 7th for 10th 21 Accepts 
1 1-2-3-4-5-6-10-8-9-7 104.6 104.6 5.00 1st for 9th 20.1 Rejects: 
Ist for 4th 3.1 Accepaas 
2 4-2-3-1-5-6-10-8-9-7 107.7 104.6 5.00 7th for 10th i} Accepassl 
3. 42-3-1-5-6-7-8-9-10 106.4 104.6 5.00 5th for 6th 5.0 Accept 
4 42-3-1-6—-5-7-8-9-10 101.4 101.4 5.00 9th for 10th 0.3 Accepaatl 
5 4-2-3-1-6-5-—7-8-10-9 101.1 101.1 5.00 9th for 10th 0.3 
6 42-3-1-6-5—7-8-9-10 101.4 101.1 5.00 2nd for 7th  -12.9 
1st for 7th 3.6 
7  7-2-3-1-6-S—4-8-9-10 97.8 97.8 5.00 6th for 7th -6.6 
7th for 8th -1.7 
8 7-2-3-1-6-5-8—-4-9-10 99.5 97.8 5.00 2nd for Sth -9.0 
2nd for 4th -0.9 
40  3-1-2-4-7-8-5-9-10-6 81.8 81.8 2.05 6th for 10th -13.4 
5th for 6th 4.5 


2nd for 5th 24.2 
9th for 10th 15.3 
9th for 10th  -15.3 
8th for 9th -0.5 


46 3-1-2-4-7-8-9-10-5-6 85.4 81.8 2.05 Sth for 10th  -16.5 
8th for 10th -15.3 
8th for 10th  -15.3 
3rd for 6th -20.8 
2nd for 9th 39,2 
2nd for 9th 39.2 
4th for 9th 22.5 
3rd for 9th 32.2 
Sth for 7th 21.3 
Ist for 3rd -3.8 
2nd for 8th 59.6 
9th for 10th 0.7 


47  3-1-2-4-7-8-9-10-6-5 84.7 81.8 2.05 3rd for 8th -52.6 
8th for 9th 9.8 
4th for Sth —0.7 


7th for 10th -12.4 
7th for 10th -12.4 
Sth for 10th -12.0 
1st for 7th -34.0 
4th for 10th  -13.3 
2nd for 10th -18.6 
6th for 10th -7.8 
6th for 7th -18.3 
3rd for Sth -12.7 
9th for 10th 0.7 


48  3-1-2-4-7-8-9-10-5-6 85.4 81.8 2.05 9th for 10th 0.7 
49 3-1-2-4-7-8-9-10-6-5 84.7 81.8 2.05 7th for 8th 0.2 
50 3-1-2-4~7-8-10-9-6-5 84.5 81.8 1.64 Stop 
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FIGURE 12.8 Solution Values in Simulated Annealing 
Search of NCB Example 


Temperatures in this simulated annealing example began at gq = 5.0. Every 10 
iterations they were reduced by a factor of 0.8, so that 


g =5.0 for iterations t = 0,...,9 
q = 0.8(5.0) = 4.0 for iterations t = 10,...,19 
q = 0.8(4.0) = 3.2 for iterations t = 20,...,29 


The first few iterations of Table 12.9 show most randomly chosen moves being 
accepted. Interchange of the seventh and tenth tour positions improves on solution 
y” and is accepted immediately to produce y™. The first move generated from 
solution y”) increases length by 20.1 inches. Thus its probability of acceptance was 


es 0.018 


Not surprisingly, it was rejected. 
The next try produced a move that increased length by only 3.1 inches. With 
better probability 


go 0.538 


it was adopted. 

The latter part of Table 12.9 shows the impact of reducing temperature g as 
the search proceeds. More and more nonimproving moves are rejected, because the 
orobability of acceptance has declined with q. 

Results in Figure 12.8 show the wide-ranging evolution of the full simulated 
annealing search. As with tabu, the final incumbent solution happens to be the 
2lobal optimum, but this is not guaranteed. 

Again this behavior is typical of reported simulated annealing applications, al- 
‘hough many more iterations would normally be required. Suitable implementations 


694 Chapter Twelve e Discrete Optimization Methods 


of the simulated annealing variation on improving search can greatly enhance th: 
quality of heuristic solutions obtained. 


SAMPLE EXERCISE 12. 32: APPLY. ING simu 


Return again to the knapsack model of Sample Bxereises 1 29 to 12.31 with : 
single-complement move set and initial solutionx® = (1, 0, 0, 0). Using tempera - 
q = 10, apply simulated annealing Algorithm 12E through tmax = 3 steps. Wher 
random decisions are required, use the following random numbers (uniform betwee 
0 and 1): 0.72, 0.83, 0.33, 0.41, 0.09, 0.94. 


Analysis: Required computations are summarized in the following table. 


Incumbent 
t x Value Value q Complement Aobj Outcome 
0 (1,0,0,0) 18 18 10 j=4 14. Accepted 
L VG@03 0) 32 32 10 j=4 -14 Rejected 
j=l -18 Accepted 
2. (0207030) 14 32 10 j=3 11 Accepted 
3° -(0;0;4,. 1) 25 32 10 Stop 


The process begins by randomly selecting between feasible moves complementins 
j =3andj = 4 atx® = (1,0,0,0). Since the first random number 0.72 is in the 
upper half of interval [0,1], 7 = 4 is provisionally selected. The corresponding mov: 
is improving, so the search advances to x = (1, 0, 0, 1) with value 32. 

Feasible complements at x are j = 1 andj = 4. The next random numbe 
0.83 selects j = 4. The corresponding move has Aobj = —14, so we test whethes 
random number 0.33 is at most probability 


mma ey trie O92 


It is not, and the provisional move is rejected. 
The next randomly generated feasible move complements j = 1 with Ao» 
= —18. This time 


ghd — e-E aOuiG see OO 


We accept the move and advance to x? = (0, 0, 0, 1). 

Three moves are feasible at x® and complementation of j = 3 is selected 
by random number 0.94. The corresponding improving move advances to x° = 
(0, 0, 1, 1). We now stop att = tmax = 3 and report incumbent solution x = (1, 0. 0. 
with value 32. 


Genetic Algorithms 


A third popular method for avoiding local optima in improving search is known & 
genetic algorithms because it attempts to parallel the process of biological evolutios 
to find better and better solutions. 
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The best single solution encountered so far will always be part of the population 
(in the variant discussed here), but each generation will also include a spectrum of 
other solutions. Ideally, all will be feasible, and some may be nearly as good in the 
objective function as the best. Others may have quite poor solution values. 

New solutions are created by combining pairs of individuals in the popula- 
tion. Local optima are less frequent because this combining process does not center 
entirely on the best current solution. 


Crossover Operations in Genetic Algorithms 


The standard genetic algorithm method for combining solutions of the population is 
known as crossover. 


ssover combine: nes a a pair of “parent” solutions to produce a pair of “chil- 
aking | both parent vectors at the same point and reassembling the — 
one parent solution with the second Part 0 of the other, and vice 


We can illustrate with two binary solution vectors x and x®: 
x = (1,0,1,1,0, | 0,1, 0,0) 
c= (Ost, 1.0). 1s | AO. O, 1) 
Crossover after component j = 5 leads to children 
a0 (10,0100, 1,00) 
x® = (0,1,1,0,1, | 0,1, 0,0) 


One child x® combines the initial part of x“ with the final part of x®. Child x 
does just the opposite. Both become members of the new population, and the search 
continues. 

There is no guarantee that crossover’s rather arbitrary manipulation of parent 
solutions will yield improvement. Still, it does lead to fundamentally new solutions 
that preserve significant parts of their parents. Experience shows that this is often 
enough to produce very good results. 


Managing Genetic Algorithms with Elites, Immigrants, 
and Crossovers 


Many variations on the basic genetic algorithm strategy have been employed success- 
fully in pene oulas applications, including many alternatives to standard crossover op- 


erations . Principal differences in the various implementations concern how 
to select pairs of current solutions to produce new ones via crossover, how to decide 
which new and/or old solutions will survive in the next population, and how to main- 
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tain diversity in the population as the search advances from generation to generat = 
The only requirement is that better solutions have greater chance to breed. 

In this brief introduction we consider only a single elitest method of populat = 
management. Each new generation will be composed of a combination of =~ 
immigrant, and crossover solutions. 


Maintenance of the elite solutions from the preceding generation assures that the == 
solutions known so far will remain in the population and have more opportun= = 
to produce offspring. Addition of new immigrant solutions will help to mainte= 
diversity as solutions are combined. The bulk of the new solutions will be the produ= 
of crossovers, with elites in the preceding population allowed to serve as pareu™ 
Algorithm 12F details a full procedure. 


ALGORITHM I2F: GENETICALGORITHM SEARCH 


Step 0: Initialization. Choose a population size p, initial starting feasib= 


solutions x", ..., x), a generation limit tmax, and population subdivisions 
Pe for elites, p; for immigrants, and p, for crossovers. Also set generation 
index t — 0. 


Step 1: Stopping. If ¢ = tmax, stop and report the best solution of the curres® 
population as an approximate optimum. 

Step 2: Elite. Initialize the population of generation t + 1 with copies of the 
Pe best solutions in the current generation. 

Step 3: Immigrants. Arbitrarily choose p; new immigrant feasible solutions 
and include them in the t + 1 population. 

Step 4: Crossovers. Choose p./2 nonoverlapping pairs of solutions from 
the generation t population, and execute crossover on each pair at om 
independently chosen random cut point to complete the generation t + ° 
population. 

Step 5: Increment. Increment ¢ <— t+ 1, and return to Step 1. 


Solution Encoding for Genetic Algorithm Search 


Just as eee we an Kou seis improving search requires careful construction of ams 


12.49 |), implementations of genetic algorithms rega~ 
ire aaied of a scheme for encoding solutions in a vector. To see the diffica™ 
return to our NCB drilling example. 
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If solutions are encoded merely by displaying the drilling sequence, two that 
might come together as crossover parents would be 


(3, 1,2, 4,7,8,5,9,10,6) and (7,2,3,1,6,5,8, 4, 9, 10) 
Crossing over the solutions after component j = 6 would yield children 
(3,1, 2,4, 7,8,8,4,9,10) and (7,2,3,1,6,5,5,9, 10, 6) 


Neither is a feasible drilling sequence because some holes are visited more than once 
and some never. A poor choice of solution encoding has made it almost impossible 
for crossover to produce useful new solutions. 


12.57 | Effective genetic algorithm search requires. choice for encoding problem — 
i 


We can obtain a better encoding in the NCB example by a technique known as 
random keys. Sequences are encoded indirectly under random keys as a vector of 
random numbers such as 


(0.32, 0.56, 0.91, 0.44, 0.21, 0.68, 0.51, 0.07, 0.12, 0.39) 


The drilling sequence implied is the one obtained by aligning hole 1 with the lowest 
random component, hole 2 at the next lowest, and so on. That is, the given random 
vector encodes the drilling sequence 


(48; 10,65 3; 95 75.2.5) 


But notice that crossover on two vectors of random numbers produces two others. 
Thus every crossover operation of these random key encodings will yield two new 
feasible solutions. Also, it is very easy to generate arbitrary new solutions for the 
initial population and immigration simply by generating random vectors of random 
numbers. 


Genetic Algorithm Search of NCB Example 


Figure 12.9 shows objective function values in a 30-generation Algorithm 12F search 
of our NCB example. The population had size 20, with the initial population gen- 
erated randomly. Following principle [12.56], each new generation contained the 
6 best (elite) solutions of the preceding generation, 4 randomly generated immi- 
grant solutions, and 10 solutions obtained from crossover of 5 pairs of parents in the 
preceding generation. All solutions were encoded by random keys. 

Bars in Figure 12.9 extend from the lowest to the highest route length of so- 
lutions in each population. Notice that the lowest ones converge systematically to 
81.8, which we know is the optimal value of this example. Still, the population always 
contains a variety of solutions, some with rather poor objective values. This diversity 
makes it possible for the search to range into distinctly new parts of the feasible space 
as it seeks improved solutions. 
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12.8 CONSTRUCTIVE HEURISTICS 


Improving search heuristics of Section 12.6 and 12.7 move from complete solut 

to complete solution. The constructive search alternative, which is the subject 
this section, follows a strategy more like the branch and bound searches of Sect 
12.4 and 12.5. They proceed through partial solutions (definition ), choosims 
values for decision variables one at a time and (often) stopping upon completion 

a first feasible solution. 


Rudimentary Constructive Search Algorithm 


Constructive searches typically begin with every discrete component of the decis 
vector free. At each iteration, one previously free variable is fixed at a value feasi> = 
with decisions fixed so far. That is, the chosen value for the new component shou” 
not produce constraint violation when previously fixed values are substituted an- 
optimistic assumptions about free variable values are adopted. 

In the simplest case, the process terminates when no free variables rema= 
Algorithm 12G gives a more formal statement. 


12.8 Constructive Heuristics 


Step 0: Initialization. Start with all-free initial partial solution 
x = (#,...,4#) and set solution index t — 0. 

Step 1: Stopping. If all components of current solution x are fixed, stop 
and output X <— x® as an approximate optimum. 

Step 2: Step. Choose a free component x, of partial solution x® and a value 
for it that plausibly leads to good feasible completions. Then, advance to 
partial solution x“*) identical to x except that x, is fixed at the chosen 
value. 

Step 3: Increment. Increment ¢ — t+ 1, and return to Step 1. 


Greedy Choices of Variables to Fix 


Obviously, the bulk of the effort in constructive searches goes to choosing the next 
free variable to fix and picking its value. Most common procedures accomplish these 
tasks in a greedy or myopic fashion. 


bas 


That is, greedy rules choose the fix that seems most likely, on the basis of what is 
presently known, to lead to a good feasible completions. 

Just as with improving search, there is risk in looking only locally for informa- 
tion about the next choice. Quite possibly a decision that appears very good with 
only a few variables fixed will actually end up forcing the search into a very poor 
part of the feasible space. Still, if the procedure is to be computationally efficient, 
compromises have to be made. 


Greedy Rule for NASA Example 


We can illustrate the idea of constructive search with the NASA capital budgeting 
model (12.16). There, the decision variables 


ap & 1 if mission j is selected 
Loe lk. O otherwise 


It is natural to construct a solution for capital budgeting models like this one 
by successively adding missions until constraints block further inclusions. That is, 
we will seek to fix a previously free x; at value 1 as long as any free mission can be 
selected without violating a constraint. 

To implement this natural constructive search we need a greedy selection crite- 
rion of the type described in F258). Certainly, we should prefer missions with high 
objective function “value” coefficients. But we also want to consider constraints. 
One high-value mission might consume so much of the various budgets that it would 
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block all further decisions. Also, precedence constraints change the implicit va! 
a mission; choosing the mission gains its value and makes successors feasible. 

Our search will tradeoff these objective and constraint considerations in 2 
mon way, by comparing missions according to the ratios 


ian allowance for . 
Prod +] enabled successor Ci ok ( =) 
j value J 2 
values r free k preceded by j (1 
— 5 ae = : 
i. s_ / fraction of remaining 8 f aij 
constraint 7 right-hand pe po 
i=1 \ side consumed by j ak 
where 


cj A objective function coefficient for mission i 
ai; & coefficient for mission j in the ith main constraint 


b? 4 right-hand side remaining in the ith main constraint after fixing 
variable values as in partial solution x® 


Among the free projects for which all predecessors have been scheduled in > 
solution x, we will fix the x; with maximum ratio 7;. If there remains room f= 
all applicable budgets, it is fixed = 1. Otherwise, we set xj = 0. 

Like many such greedy indices, ratio (12.23) seems rather complicated = 
glance, The numerator tries to account for both the immediate value of selec* 
mission, and the potential it opens up to select missions of which it is a predece 
Half the value of all enabled successors is arbitrarily added to the mission’s = 
value. The denominator of (12.23) sums the fractions of remaining row “resou: 
that a mission would consume if selected. Thus we favor missions using relat 
little of now-scarce resources. 

Ratioing value to resource use combines objective and constraint cons? 
tions. The highest 7; will correspond to a mission j high in value, or low in rese 
consumption, or both. Selecting that mission may not be the best long-term dec* 
but it does reflect about all we can know without looking more than one step 
the future. 


columns or subsets that together include or cover every element of a given set. — 
example is 


min 15x; +18x. +6x3 +20x4 


st. + xy SP ete IL 
alee kitty alma + x >1 
liga hater 08) ala ae 
ib hater OOK 


Explain why it would make sense to choose free x; to fix = 1 by picking one » 


12.8 Constructive Heuristics 701 


least ratio 
cost coefficient of column j 


yp iA 

bien number of uncovered elements that j covers 

Analysis: The proposed ratio explicitly seeks minimum cost by including the objec- 
tive function coefficient in its numerator. Still, it also considers feasibility in dividing 
by the number of still uncovered rows or elements each free j could resolve. The ef- 
fect is to seek the most efficient next choice of x; to fix = 1, the best in the short-term 
or myopic sense. 


Constructive Heuristic Solution of NASA Example 


Starting from the completely free partial solution 
Ce a ee ee | 
all oy equal initial right-hand sides. Ratios for the first two j’s are 


200 
n= 6/10 = 333.33 
the itera 
2 = Gin Ga 


and similar arithmetic yields 
P= MOOT far 29.17 5° ts = 35.21, 16 = 21,54 
t7-=3O2, rg = 7.37, ro = 110.09, rig = 157.50 
ry = 10.96, m2 =7.38, m3 = 586.05, 4 = 87.30 


The highest of these ratios is 586.05 for mission 13. Since this mission fits within 
remaining right-hand sides b!” and has no predecessors, we fix x13 = 1 to produce 


xO-[ GAEL EEEREBEEE LD 


Table 12.10 provides an abridged summary of the rest of the search. Processing 
of ¢ = 1 parallels the first iteration, selecting and fixing = 1 additional mission 1. 

Something different occurs at t = 3. The mission with the maximum 7; there 
is number 9. But mission 9 requires $5 billion in the 2000-2004 budget period, and 
projects already chosen use all but be ) = $3 billion. We have to fix xX 9 = 0 to maintain 
feasibility. 

Another peculiarity arises at iteration t = 9. There the mission with the best 
ratio is j = 11. However, mission 11 cannot be selected before predecessor mission 
2. Thus we pass to the second best ratio, which happens to be j = 2, and fix x. = 1. 

Our constructive search terminates when all 14 components of the decision 
vector have been fixed. The heuristic optimal solution produced is 


¥ 4x" = (1,1,0,0,0, 0,0, 0, 0, 1,1, 0,1, 0) 


flying missions 1, 2, 10, 11, and 13 at a total value of 671. (Compare with the branch 
and bound results in Table 12.3.) 


702 Chapter Twelve e Discrete Optimization Methods 


TABLE 12.10 Constructive ‘Search of NASA Example 


t Computation Choice 
0 xO-@# 4 tet EELEEBED 

6 =10, BM =12, OM =14, A =14, 

pM =14, BM =1, MM=1, =1 


7, = 333.33, 72 = 26.67, = r3= 129.07, r4= 29.17, Select j = 13 

5 = 35.21, te = 2154, © re'= 6952; rg = 7.37, and fix x13 = 1 
rg = 110.09, 749 = 157.50, ry, = 10.96, ry = 7.38, 

713 = 586.05, r4 = 87.30 


1 xO-G@4 #4448548 44 18) 
Om @_ @_ O 
oie ef 1, a 10, He 13, 
DBS WMS tat, oe) =1 


r, = 333.33, 2 = 25.38, r3=122.59, r4= 28.26, Select j =1 
r5= 31.05, r6=19.55, 1r,=6.05, rg = 7.22, and fix x, = 1 
rg = 107.84, 49 = 133.06, 4, = 10.35, ry = 7.04, 

73 = N/A, ri4>= 79.56 


3 xO=(0, 44444488 .1,48,19 
pb =4, B=3, B=6, 10 =13, 
(3) _ (3) _ (3) _ (3) _ 
Be 3 pee be ge ol 


r,=N/A, ry = 8.00, 73 = 38.28, rg = 28.26, Select j =9 
t5=17.50, 76=18.35, r,=1.71, rg = 7.22, and fix x9 = 0 
79=54.54,  Mo=N/A, 7, =9.61, 42 =2.23, because violates 
rg=N/A, — r44= 50.99 contraint 2 


9 x) = (4, #, 0,0, 0, 0, #, #, 0, 1, #, #, 1, 0) 
9) _ oy eal Oem 
br) 13,0 Bee 1g Bl bead 


r,;=N/A, ry = 8.00, r,=N/A, r4=N/A, Select second best 

t5=N/A, 16=N/A, r= 1.71, 7g = 7.22, j=2 and fix 

ry = N/A, Tyo= N/A, ry = 9.61, yp = 2.23, X) = 1 because 

ry3 = N/A, ry4=N/A j= 11 has free 
predecessor 


10 x = (1,1, 0,0, 0, 0, #, #, 0, 1, #, #, 1, 0) 
BM =2, BM =0, BM =6, 20% =13 
BO =13, H=1, M7, p24 


r,=N/A, 1, = N/A, r,=N/A,  1=N/A, Select j = 11 

- Ts = N/A, 16=N/A, r; = 0.0000, rg = 0.0001, and fix x;; =1 
Tg = N/A, Tyo=N/A, 4, = 0.002, 42 = 0.0000, 
ry3 = N/A, y4=N/A 


14. x4) = (1,1,0,0,0,0,0,0, 0, 1, 1,0, 1,0) 
EEE ORE RE NETSE LOC enieices eee aes Be ee 
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SAMPLE EXERCISE 12.34 G CONSTRUCTIVE HEURISTICS = 


Return to the set covering problem of Sample Exercise 12.33 and its sugested mieedy 
ratio r;. Use this ratio to apply Algorithm 12G, fixing selected x; = 1 as long as rows 
remain uncovered and = 0 thereafter. 


Analysis: The search begins with all-free partial solution x = (#, #, #, #). All rows 
of the set covering model are uncovered, so ratios compute 


15 Sls) 6 20 


TB" syrgee D  gaLs e 


The least of these values occurs at j = 3, so we fix x3 to obtain x = (#, #, 1, #). 


The third row of the model is now satisfied. This leads to revised ratios 


aS 18 20 
Wp e Sony A De ayia. AL “oy 


Choosing the least fixes x; inx® = (1, #, 1, #). 

All rows are covered by partial solution x, Thus the least cost choice for 
remaining components is zero. We stop with heuristic optimum X = (1,0, 1, 0) at 
cost 6 + 15 = 21. Notice that this solution is not as good as optimal x* = (0, 0, 0, 1) 
with cost 20. 


Constructive Search as the Method of Last Resort 


Many project selection and capital budgeting models are approached by greedy con- 
structive heuristics exactly like the one just illustrated. Still, we have seen in Section 
12.5 that more exact, branch and bound methods can also be effective. 

The real need for constructive search methods becomes clear only with large, 
often nonlinear, highly combinatorial discrete models such as the KI truck routing 
problem of Section 11.5 or cases where we need an answer fast. 


finding ¢ good solutions. 


If tractable and strong relaxations are available, branch and bound is preferred. 
When natural neighborhoods exist, improving search can be effective. If neither 
applies, constructive heuristics provide the method of last resort. 


Constructive Search of KI Truck Routing Example 


To illustrate constructive search in such highly combinatorial cases, we will develop an 
algorithm for the KI routing example (Section 11.5). Recall that stopsi=1,..., 20 
are to be organized into the smallest possible list of routes j originating and terminat- 
ing at a single central depot. Each route is then sequenced by an improving search to 
minimize travel distance. Figure 12.10 shows stop locations, and Table 12.11 provides 
the fractions of a load to be delivered at each stop. 

Our constructive search for KI begins each route with a “seed” stop. We will 
choose the free stop farthest from the depot—number i = 9 in the first route. Figure 
12.10 shows that the idea is to create a starting, out-and-back route with a general 
direction anchored by the seed location. 
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1@ @3 ze 7® 

2® insertion to 88 
complete 
a route} 


5@r, 


@10 


initial center 
of gravity 


initial out and 


6® back route 
| | ene 
depot 18° 20 
e 

e 14 7 

15 [ee 
e FIGURE 12.10 Locations and First Rout< 
20 in KI Example 


As long as capacity remains in the truck for a route, we will next insert m=» 
stops. As usual, our approach is greedy. A “center of gravity” is computed for si» 
so far fixed into the route, and the closest stop to that center still fitting on the tras 


is added to the route. 


Figure 12.10 shows that a 1-stop route’s center of gravity is set arbitrarily ® 
of the way from the depot to the seed location. This initial center of gravity for 


route started by stop 9 has coordinates 
0.8(x9, yo) = 0.8(15, 20) = (12, 16) 


We hope to grow a cluster of stops near that point to form a compact route. 


Stop 9 already uses fo = 0.50 truck. The nearest stop to the center of gra 
is i = 8, with load fg = 0.43 within the remaining capacity. It becomes the 


insertion. 


After the route has more than one stop, the new selection is averaged int 


center of gravity as 
- [0.8(15, 20) + (11, 17)] = (11.5, 16.5) 
However, the capacity fixed on this route already sums to 
fg +fo = 0.50 + 0.43 = 0.93 


and no remaining load will fit in the residual 0.07 truckload capacity. Route j = 


complete. 


The next seed location is the farthest free stop from the depot—number i = 
In turn, stop i = 7 is fixed in the route, then stop 7 = 11, and finally i = 5. These fos 


TABLE 12.11 Fractions of Truckloads to Be Delivered in KI Example 


Stop,i Fraction, f, | Stop,i Fraction, f, 
6 11 


Stop, i Stop, i 


1 0.25 0.21 16 0.38 
2 0.33 12 0.68 17 0.26 
3 0.39 13 0.16 18 0.29 
4 0.40 14 0.19 19 0.17 
5 0.22 


Fraction, 7 
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locations exhaust 0.98 of capacity, leaving no room for more. 
Continuing in this way produces the full route structure of Figure 12.11. A total 
of 7 routes were constructed, seeded byi = 9, 10, 1, 20, 19, 16, and 18, respectively. 


20 


FIGURE 12.11 Final Routes in KI Example 


EXERCISES 


12-1 Solve each of the following discrete optimiza- 
tion models by total enumeration. 


(a) &] min 2x, +x, + 4x3 + 10x, 
s.t. Xy tx. +43 <2 
3x1 + 7x2 + 19x3 + x4 > 20 
X1,%2,%3 = O0orl 
xe =O 


bo) max 30x; + 12x + 24x; + 55x4 
s.t. 30x, + 20x2 + 40x3 + 35x4 < 45 
Xo + 2x3 +%4 > 2 
x, >0 
X2,%3,xX4 =O0orl 


12-2 Suppose that you have been asked to solve a 
mixed-integer ILP with 10,000 continuous and n bi- 
nary decision variables. Determine the largest n’s 
for which the problem could be totally enumerated 
‘n one 24-hour day and in one 24-hour, 30-day month 
>y each of the following computer environments. 


a) An engineering workstation that can enu- ~ 


merate one choice of binary variables each sec- 
ond, including solving the resulting LP. 


(b) A parallel processing computer that can evalu- 
ate 32,768 choices simultaneously every second, 
including solving the resulting LPs. 


12-3 Consider the ILP 
max 14x; + 2x. —11x3 + 17x4 
s.t. 2x1 +xX2 + 4x3 + 5x4 < 12 
Xx, — 3x2 — 3x3 — 3x4 <0 
x, >0 
X2,X3,xX4 =Oorl 
Determine whether each of the following is a con- 
straint relaxation. 


(a) RB) max 14x, + 2x — 113 +17x4 
s.t. 2x, +X2 + 4x3 4+ 5x4 < 12 
X1 — 3x2 — 3x3 — 3x4 <0 
GSO f= law 
(b) max 14x, +2x. —11x3 + 17x, 
S.t. 2x, +X. + 4x3 + 5x4 < 12 
X1 — 3x2 — 3x3 — 3x4 <0 
x; > 0 and integer 
xX2,%3,xX4 =Oorl 
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(c) ) max 14x; + 2x2 — 11x3 + 17x4 
s.t. 2x, +x. + 4x3 + 5x4 <5 
X1 — 3x2 — 3x3 — 3x4 < 0 
x, >0 
X2,X3,X4 = Oorl 


14x, + 2x. — 11x3 + 17Xx4 
s.t. X1 — 3x2 — 3x3 — 3x4 <0 
w= 0 

X2,X%3,X4 = Oorl 


12-4 Form the linear programming relaxation of 
each of the following ILPs. 


(a) [] min 12x, + 45x. + 67x3 + 1x4 
S.t. 4x, + 2x2. —x4 < 10 
6x; + 19x3 > 5 
X2,X3,X4 = 0 
x; =Oorl 
x3 integer 


(b) max 9x; + 3x2 + 4x3 + 8x4 
s.t. 2x1 + 2x2 + 2x3 + 3x4 < 20 
29x, + 14x. + 78x3 + 20x4 < 200 
X1,X2,x3 =Oorl 
x =0 


12-5 Each of the following ILPs has no feasible solu- 
tions. Solve the corresponding LP relaxation graph- 
ically and indicate whether your relaxation results 
are sufficient to show that the ILP is infeasible. 


(a) B) min 10x; + 15x. 
s.t. Xp +% >2 
—2x, +2x, >1 
X1,X2 =Oorl 
(b) max 11x; + 21x. 


SiG: 2x1 +X. > 2 
2x, —X, <0 
X1,X2 =Oorl 


(c) Kes} min 2X4 =Fity 
s.t. xX, + 4x2 <2 
—4x, ae 4x Zeb 


xX, >0, x» =Oorl 
(d) max 47x; + 58x2 
s.t. x+%x%>4 
x, =Oorl 
0< x2< 2 


12-6 Determine the best bound on the optimal so- 
lution value of an ILP with each of the following ob- 
jective functions that is available from the specified 
LP relaxation optima x. 
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(a) &] max cee + 13x2 + 3x3 


X= (250) 
(b) min x; — 4x2 + 70x3 
x= (1,0, 2) 
(c) B&Q] min 60x, — 16x2 + 10x3 
& = (3,1, 5) 
(d) max 90x; + 11x. + 30x3 
« = (0, $.5) 


12-7 Determine whether each of the follows. ~ 
relaxation optima x is optimal in the correspos 
ILP over the specified variable type constrain’ 


(a) Oy ae cei 
= (1,0, 4,2) 
(b) ae tant X3,xX4 >0 
Ke (Onl 5.5) 
(c) I) x1,x%2,x3 = 0 orl, x4 = 0 
= (1,0 2) 
(d) x; > Oand integer,j=1,...,4 
2 = (0.32.1) 
12-8 The ILP- 
max 3x; +.6x2 + 4x3 + 10x4 + 3x5 
st. 2x, + 4x5 +23 + 3x4 + 7x5 < 10 
X1 +x3+xX4 <2 
4x + 4x4 + 4x5 <7 
Mines =O ord 


has LP relaxation optimal solution * 


(Of0275, 1-150); 


(a) [S| Determine the best bound on the ILP ~ 
timal solution value available from relaxes ~ 
results. 

(b) [§] Determine whether the relaxation op 
solves the full ILP. If not, round to an ~~ 
feasible solution either by moving all >= 
variables at fractional values in the relaxes 
up to 1 or by moving all down to 0. 

(c) [| Combine parts (a) and (b) to determin: 
best upper and lower bounds on the ILPom = 
solution value available from the combins: ~ 
of relaxation and rounding. 

(d) [| 2 Verify your bounds of part (c) by ss 
the full ILP with class optimization softw=- 


12-9 Do Exercise 12-8 for the ILP 


min 4x, 5x2 t 12x3 + 7X4 + OX5 
s.t. 6x, + 8x. + 21x3 + 6x4 5x5 > 11 
Xy +X. +2x3+4%,>1 


me + 5x3 +%5 > j 
eS 00 
and LP religations optaruat pa = (0, 0, 0.533. 0 


es eee ee ee ee ee 


12-10 [| Do Exercise 12-8 for the ILP 


min 17x; + 12x. + 24x3 + 2x4 + 8x; 
st. 3x, + 5x3 + 7x4 + 9x5 > 13 
7x2 + 4x4 +11x5 > 5 
2x1 + 3x2 +2x3+3x4>7 
X2,X3,X4 = Oorl 
X25 220 


and LP relaxation optimum x = (0.5, 1, 0, 1, 0.5). 
12-11 Do Exercise 12-8 for the ILP 


min 40x3 + 500x4 + 800xs + 900x¢ 
s.t. 10x, + 6x2 + 2x3 = 45 
2x, + 3x2 +x3 > 12 
O <x, < 5x4 
0 <x < 5x5 
0 < x3 < 5x5 
M4. 55X65 Ore! 
and LP relaxation optimum x = (3.500, 1.667, 
0, 0.70, 0.333, 0). 
12-12 Consider the ILP 
min 10x; + 20x. + 40x3 + 80x4 — 144y 
St. xy +X. +%3 +2X4 > 4y 
Baars ij Oto} gall 


(a) [] Solve the full ILP model by inspection. 

(b) Verify by inspection that its LP relaxation has 
optimal solution ¥ = (1,1, 0,0), 7 = 4. 

(c) [| Show that an equivalent ILP would result if 


the main constraint were replaced by 
xj =y Fi lien 


(d) [] Verify that the revised formulation of part 
(c) has a stronger LP relaxation than the origi- 
nal of part (b). 


12-13 Do Exercise 12-12 for ILP 


min 14x, + 16x. + 15x3 
St. x +%,>1 

x +x32>1 

xX, +x32>1 

My neg ior I 


LP relaxation optimum xX = 
main constraint 


(3, 4.) and revised 


Xi, +X. +%x3>2 


12-14 Return to the ILP of Exercise 12-12 with LP re- 
laxation optimum x = (1,1, 0, 0), j = $. Determine 
whether each of the following is a valid inequality 
for the ILP, and if so, whether it would strengthen 
the original LP relaxation to add the inequality as a 
constraint. 
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(a) x2 +x3 +x4 > 3y 

(b) BO] xy +2 +43 +24 > 4y 
() Mate e1 

d@) Rlxssy 


12-15 Return to the ILP of Exercise 12-13 with LP re- 
laxation optimum x = (3, +, 4). Determine whether 
each of the following is a valid inequality for the ILP, 
and if so, whether it would strengthen the original 


LP relaxation to add the inequality as a constraint. 


(a) 10x, + 10x2 + 10x3 > 25 
(b) x, +%2+%32>1 
(c) X1 +X. +x3>2 
(2) 14x; + 20x) + 16x3 > 28 


12-16 The ILP 


max 40x; + 5x2 + 60x3 + 8x4 
s.t. 18x, + 3x2 + 20x3 + 5x4 < 25 
X13. er Vor! 

has LP relaxation optimum X = (+, 0, 1, 0). Deter- 
mine whether each of the following is a valid inequal- 
ity for the ILP, and if so, whether it would strengthen 
the LP relaxation to add the inequality as a con- 
straint. 


(a) Bl x+x3<1 

(b) Xp tx. +x34+%x4 <4 
(c) DB mt+xy>1 

(d) 18x; + 20x; < 20 


12-17 The fixed-charge ILP 


min 60x; + 78x. + 200y, + 400y2 
S.t. 12x; + 20x2 > 64 

15x; + 10x. < 60 

x, +x2 < 10 

O<x,< 100y, 

O<%m < 100y2 

yi, y2 =Oorl 


has LP relaxation optimum x = (0,3.2), 7 = 
(0, 0.032). 


(a) [| Compute the smallest replacements for big- 
M values of 100 in this formulation that can be 
inferred simply by examining constraints of the 
model. 

(b) [] Show that the LP relaxation optimum will 
change if the lower big-M’s of part (a) are em- 
ployed. 

(c) &] & Verify part (b) by solving the model hav- 
ing smaller big-M’s with class optimization soft- 
ware. 
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12-18 Do Exercise 12-17 for the average-comple- 
tion-time, single-machine scheduling ILP 


min 0.5(x, +12 +2 +8) 

st. x, +12 <x,+100(1 - y) 
Xo +8<x,+100y 
X1,X. > 0 
y=0Oorl 


with LP relaxation optimum x = (0,0), y = 0.08. 
[Hint: Consider the sum of the process times in part 
(a).] 
12-19 Consider the ILP 
max 30x, +55x2 + 20x3 
st. 40x, — 12x, + 11x; < 55 
19x; + 60x2 + 3x3 > 20 
3x, + 2x2 +2x3 = 5 
X1,X2,%3 =Oorl 


Form the Lagrangian relaxations obtained by dual- 
izing each of the following collections of main con- 
straints, and show all sign restrictions that apply to 
Lagrange multipliers. 


(a) [Q] Dualize the first and second main con- 
straints. 
(b) Dualize the second and third main constraints. 


12-20 Consider the facilities location ILP 


min 3x11 + 6x1,2 + 5x2.) + 2x22 
+250y; + 300y2 

St. 30x11 + 20x12 < 50y, 
30x2,1 + 20X22 < 50y2 
41 +%21=1 
X12 $22 =1 
0 < X11,%1,2,%21,%2,2 <1 
y; =Oorl 


(a) [§] Use total enumeration to compute an opti- 
mal solution. 

(b) [S| Form a Lagrangian relaxation dualizing the 
third and fourth constraints with Lagrange mul- 
tipliers v; and v2. 

(c) [Q] Explain how the dualization in part (b) 
leaves a relaxation that is easier to solve than 
the full ILP. 

(d) EK] Use total enumeration to solve the La- 
grangian relaxation of part (b) with v; = v2 = 0, 
and verify that the relaxation optimal value pro- 
vides a lower bound on the true optimal value 
computed in part (a). 

(e) Repeat part (d) with v; = v = 100. 
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(f) EK] Repeat part (d) with v; = 1000, v2 = 500. 


12-21 Do Exercise 12-20 for the generalized assig= 
ment model 


min 15x11 + 10x12 + 30x24 + 20x22 
s.t. Dei toe 

X21 +X22 =1 

30x11 + 50x24 < 80 

30x12 + 50x2,2 < 60 

X1,15 1,2, 2,1, X2,2 = 0 or 1 


Dualize the first two main constraints and solve w 
v = (0,0), v = (10, 12), and v = (200, 100). 

12-22 Suppose that an integer linear program bh: 
decision variables x1,%2,x3 = Oor1. List all com 
pletions of the following partial solutions. 


(a) [l @#,0,#) 
(b) 1, 0) 
12-23 The following is the complete branch-an- 


bound tree for an ILP over decision variabd = 
Xiph Olor ik 


(a) B&Q] List the partial solutions associated we 
each node of the tree. 

(b) [|] Which nodes were branched and which te 
minated? 

(c) [Q] Identify the nodes of the tree that ha: 
x = (0,1, 0,1) as a feasible completion. 


12-24 Do Exercise 12-23 for the branch and bous- 
tree 


12-25 Suppose that the ILP of Exercise 12-8 is be- 
ing solved by branch and bound. State the candidate 
problem associated with each of the following partial 
solutions. 


(a) [O] (#1, #, #, 0) 
(b) (0,1, #,1, 0) 


12-26 Suppose that a minimizing ILP is being solved 
by LP-based branch and bound Algorithm 12A over 
decision variables x;,x2,x3 = Oor1, x4 > 0. Show 
how the search should process the node with x. = 1 
and other variables free if the corresponding LP re- 
laxation has each of the following outcomes. Assume 
that the incumbent solution value is 100. 


(a) [| & = (.9, 1, 0, 6), value i = 97 

(b) ¥ = (0.2, 1,0.77, 4.5), value = 116 
(c) Bl X=, 1,0, 4.2), value 5 = 75 
(d) LP relaxation infeasible 

(ec) &] X= (1,1, 0.6, 0), value > = 100 
(f) ¥= 4,1, 0.1, 5.9), value 5 = 83 


12-27 [6] The following table shows the LP relax- 
ation outcomes for all possible combinations of fixed 
and free variables in branch and bound solution of 
a minimizing integer linear program over decision 
variables x1, x2,x3 = 0 or 1, x4 > 0. Solve the prob- 
lem by LP-based Algorithm 12A and record your 
results in a branch and bound tree. Apply the depth 
first rule for selecting among active nodes and pick 
whichever of = 0 and = 1 is closest to the preced- 
ing relaxation value when nodes have equal depth. 
Branch on the integer-restricted variable with frac- 
tional relaxation value nearest to integer. 


xy x, X x v 

# # # (0,0.60,0.14,0) 60.9 
# # 0 (0.20, 0.60,0,0) 61.0 
# # 1 (0.60, 0, 1, 0) 69.0 
# 0 # (0.60, 0, 1, 0) 69.0 
# 0 0 Infeasible — 
# 0 1 (0.60, 0, 1, 0) 69.0 
# 1 #  (0,1,0,400) 4090.0 
# ut 0 (0,1,0,400) 4090.0 
# cE dl Infeasible — 
0 # # (0,0.60,0.14,0) 60.9 
0 # 0 (0, 0.60, 0, 1.9) 73.6 
0 # 1 (0, 0, 1, 6) 108.0 
0 0 # (0, 0, 1, 6) 108.0 
0 0 0 Infeasible — 
0 0 1 (0, 0, 1, 6) 108.0 


(continued top of next column) 
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(continued) 

XX 1% x v 

0 1 # = (0,1,0,400) 4090.0 
0 1 0 (0,1,0,400) 4090.0 
0 1 1 Infeasible — 
1 # # = (1, 0.33, 0, 0) 65.0 
1 # 0 (,033,0,0) 65.0 
1 # al (1, 0, 1, 0) 83.0 
1 0 # = (1,0, 0.71, 0) 69.3 
1 0 0 Infeasible —_— 
it 0 1 (1, 0, 1, 0) 83.0 
a 1 # (1,1,0,400) 4125.0 
if 1 0 (1,1,0,400) 4125.0 
1 1 1 Infeasible —_ 


12-28 Do Exercise 12-27 for a maximizing ILP over 
xX, > 0, X2,X3,X4 = O or 1 with the following LP re- 
laxation outcomes. 


X, =X; XY K v 

# # # (48.1, 0.20, 0.77,1) 78.0 
# # 0 (0, 1, 1, 0) 59.0 
# # 1 (48.1, 0.20, 0.77,1) 78.0 
# 0 # (29.6, 0.20, 0, 1) T12 
# 0 0 (0, 1, 0, 0) 34.0 
# 0 1 (29.6, 0.20, 0, 1) Tie 
# 1 # (41.6, 0.43,1,0.77) 75.0 
# 1 0 (0, 1, 1, 0) 59.0 
# 1 1 Infeasible — 
0 # # (42.5, 0, 0.77, 1) 76.8 
0 # 0 (0, 0, 1, 0) 25.0 
0 fio neal (42.5, 0, 0.77, 1) 76.8 
0 0 # (24, 0, 0, 1) 76.0 
0 0 0 (0, 0, 0, 0) 0.0 
0 0 1 (24, 0, 0, 1) 76.0 
0 1 # (29.6, 0, 1, 0.77) 72.4 
0 1 0 (0, 0, 1, 0) 25.0 
0 1 1 Infeasible = 
1 # # (12, 1, 1, 0.20) 67.0 
1 # 0 (0, 1, 1, 0) 59.0 
1 # 1 Infeasible — 
1 0 # (0, 1, 0, 0.20) 54.0 
1 0 0 (0, 1, 0, 0) 34.0 
1 0 1 Infeasible — 
1 1 # (12, 1, 1, 0.20) 67.0 
1 if 0 (0, 1, 1, 0) 59.0 
1 1 al Infeasible — 


12-29 Students often mistakenly believe ILPs are 
more tractable than LPs because the straightforward 
rules of Algorithm 12A seem less complex than the 
simplex and interior point methods of Chapters 5 
and 6. 


(a) Explain why solution of any ILP by LP-based 
branch and bound always takes at least as much 


710 


work as a linear program of comparable size and 
coefficients. 

(b) Justify why the number of LP relaxations that 
might have to be solved in branch-and-bound 
enumeration of an ILP with n binary variables 
is 20+) _ 1, 

(c) Use part (b) to compute the number of linear 
programs that could have to be solved in branch 
and bound search of ILP models with 100, 500, 
and 1000 binary variables respectively, and de- 
termine how long each such search could take 
at the rate of one LP per second. 

(d) How practical is it be to solve LPs of 100, 500, 
and 1000 variables in reasonable amounts of 
time? 

(ec) Comment on the implications of your analysis 
in parts (a) to (d) for tractability of LPs versus 
ILPs of comparable size. 


12-30 In most applications, LP based branch and 
bound Algorithm 12A actually investigates only a 
tiny fraction of the possible partial solutions. Still, 
this is not always the case. Consider the family of 
ILPs of the form 


min y 
s.t. 2) xt+y=n 
j=l 
x; =Oorl fa=Neusin 
y=0Oorl 


where n is odd. 


(a) © Enter and solve versions for n = 7, n = 11, 
and n = 15 with class branch and bound soft- 
ware, and record the number of branch and 
bound nodes explored. (Warning: Program lim- 
its must be set big enough to allow up to 20,000 
branch and bound nodes.) 

(b) Express your results in part-(a) as fractions as 
the total number of nodes that might have to be 
investigated. [Hint: Use the formula in Exer- 
cise 12-29(b).] 

(c) Comment on the implications for tractability 
of ILP’s via branch and bound if fractions like 
those of part (b) were typical. 


12-31 [S] The branch and bound tree that follows 
records solution of the knapsack model 
min 90x; + 50x. + 54x3 
st. 60x; + 110x. + 150x; > 50 
X1,%2,x3 =Oorl 
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by LP-based Algorithm 12A under rules of Exeres: 
12-27. 


(1, 0, 0) infeasi™ 


90 


Briefly describe the processing, including how a- 
why nodes were branched or terminated, when = 
cumbent solutions were discovered, and what sols 
tion proved optimal. Assume that there was no i= 
tial incumbent solution. 


12-32 Do Exercise 12-31 for 


max 51x; + 72x. + 41x; 
s.t. 17x, + 10x. + 14x3 < 19 
X1,X2,x3 =Oorl 


and tree figure top of next column 


12-33 Return to the knapsack problem of Exercis: 
12-31. 


(a) Explain why LP relaxation optimal solutio= 
can be rounded to integer-feasible solutions > 
setting X; <— [Xx]. 

(b) [| Repeat the branch and bound computatios 
of Exercise 12-31, this time rounding up each re 
laxation solution in this way to produce earlics 
incumbent solutions. 

(c) Comment on the computational savings wnt 
rounding. 


12-34 Do Exercise 12-33 on the knapsack mode! 
Exercise 12-32, this time rounding solutions % — 
[xj 

12-35 Determine the best lower and upper boune 
on the ultimate ILP optimal available from pares 
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(0.53, 1, 0) 
99 


(1, 0.2, 0) 
65.4 


io umall = x3=0 


infeasible 


x3=1 


infeasible (1, 0, 0) infeasible (0, 0, 1) 
51 41 


FIGURE 12.12 Branch and Bound Tree for Exercise 12-32 


bounds and incumbent solutions after processing (a) [Q] Show which unexplored nodes could be im- 


each node in the following: mediately terminated by parent bound if the in- 
; cumbent at node 4 had objective function value 

(a) [6] The branch and bound tree of Exercise 205. How about 210? 
12-31. (b) [] Determine the best upper bound on the ul- 


(b) The branch and bound tree of Figure 12.12. timate ILP optimal value that is available after 


processing of node 4. 

Ke Assuming the incumbent at node 4 has ob- 
jective value 195, compute the maximum abso- 
lute and percent objective value error in accept- 
ing the incumbent as an approximate optimum. 


12-36 The branch and bound tree that follows shows 

the incomplete solution of a maximizing ILP by LP- (c) 
based Algorithm 12A, with numbers next to nodes 
indicating LP relaxation solution values. 


12-37 The branch-and-bound tree that follows shows 
the incomplete solution of a minimizing ILP by LP- 
based Algorithm 12A, with numbers next to nodes 
indicating LP relaxation solution values. 


by solving 


by solving oe 


Node 4 has just produced the first incumbent solu- i 
tion, and nodes a to d remain unexplored. 


TZ 


Node 4 has just produced the first incumbent solu- 
tion, and nodes a to d remain unexplored. 


(a) Show which unexplored nodes could be imme- 
diately terminated by the parent bound if the in- 
cumbent at node 4 had objective function value 
120. How about 118? 

(b) Determine the best lower bound on the ultimate 
ILP optimal value that is available after process- 
ing of node 4. 

(c) Assuming that the incumbent at node 4 has ob- 
jective value 130, compute the maximum abso- 
lute and percent objective value error in accept- 
ing the incumbent as an approximate optimum. 


12-38 Repeat Exercise 12-27, following the same 
rules except: 


(a) [&] Use the best-first enumeration sequence and 
allow termination by parent bounds. 

(b) [Q] Use the depth-forward best-back enumera- 
tion sequence and allow termination by parent 
bounds. 


12-39 Do Exercise 12-38 for the branch and bound 
of Exercise 12-28. 


12-40 Three company trucks must be assigned to 
pickup 7 miscellaneous loads on the way back from 
their regular deliveries. Truck capacities and load 
sizes (cubic yards) are shown in the following table, 
together with the extra distance (in miles) that each 
truck would have to travel if it is to deviate to pick 
up any of the loads. 


Distance for Truck: Read 
Load 1 2 3 «Size 


1 23 45 50 4 
Z 25 12 23 8 
3 29 13 41 13 
4 12 23 40 31 
5) 49 7 42) 14 
6 37 39 59 9 
i 2, 9 PLU Al 


Capacity 30 40 50 


(a) Formulate this problem as a generalized as- 
signment ILP using the decision variables (i = 
il 2a, ype aa ) 


a | 1 ifload i goes to truck j 
0 otherwise 


(b) Enter and use class optimization software to 

compute an optimal solution. 

(c) © Use class optimization software to solve the 
corresponding LP relaxation and verify that 
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the relaxation optimal value provides a lowe 
bound. j 

(d) Using class optimization software to solve re 
laxations, verify your ILP optimal solution 5 
executing LP-based branch and bound Algo- 
rithm 12A including parent bounds. Apply the 
depth-first rule for selecting among active nodes 
and pick whichever of = 0 and = 1 is closest 
the preceding relaxation value when nodes hav< 
equal depth. Branch on the integer-restrictec 
variable with fractional relaxation value neares 
to integer picking the one with least subscript : 
there are ties. Record incumbent solutions on) 
when an LP relaxation comes out integer (ic 
do not round). 

(e) Determine when your search of part (d) cowl 
have been stopped if we were willing to ac 
cept an incumbent solution no worse than 25 
above optimal. 

(f) E\Do the same branch and bound computatior 
as part (d) except with the best first enumere- 
tion rule using LP relaxation values as paren’ 
bounds. 

(g) ©)\Do the same branch and bound computatior 
as part (d) except with the depth forward bes 
back enumeration rule using LP relaxation va - 
ues as parent bounds. 

(h) Compare your results in parts (d), (f) and (g). 


12-41 [6] The following tree records solution of = 
maximizing ILP over x),%2,x3 = Oor1 by branc® 
and cut Algorithm 12B. LP relaxations solutions 
show next to each node. 


infeasible infeasible 


Briefly describe the processing, including how and 
why nodes were branched tightened or terminated, 
when incumbent solutions were discovered, and 
what solution proved optimal. Assume that all 
added inequalities are valid for the original ILP. 
12-42 Do Exercise 12-41 for the following tree of a 
minimizing ILP over x;, x,x3 =0Oor 1. 


(0.2, 0, 0.6) 50 


X_ + 2x3 22 


(0.2, 0.5, 0.75) 57 
Xi + Xz 21 


(0.6, 0.4, 1) 58 


x,=0 


(4) (0,1, 0.5) 61 


3X2 + 3x3 25 
(0, 1, 0.67) 65 


12-43 Consider solving (approximately) the ILP 


max 5x; + 7x2 — 2x3 
SiC. Xg+x%3<1 
X1,X2,x3 =Oorl 


by a version of discrete improving search Al- 
gorithm 12C that employs move set M = 
(1,0, 0), (0, 1, 0), (0, 0, 1)} and always advances to 
the feasible neighbor with best objective value. 


a) [Q| Identify a global optimal solution by inspec- 
tion. 

5) [QJ List all points in the neighborhood of feasi- 
ble solution x = (0, 0, 1). 

c) Compute a local optimal solution by apply- 
ing Algorithm 12C starting from x = (0, 0, 1). 

4) [9] Repeat part (c), this time starting at x = 
(1, 0, 0). 
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Exercises 


(e€) Compare results in parts (c) and (d), and com- 
ment on the effect of starting solutions. 
(£) [| Repeat part (c), this time using the single- 
_ complement move set that allows any one x; = 1 
to be switched to = 0, or vice versa. 
(g) Compare results in parts (c) and (f), and com- 
ment on the effect of move set. 


12-44 Repeat Exercise 12-43 for the ILP 


min 2x; — 23x + 14x; 
s.t. Xp tx. +%3>1 
X4,%X2,x3 = Oorl 


12-45 Consider solving (approximately) the ILP 


max 12x) + 7x2 + 9x3 + 8x4 
s.t. 3x, +X. +5344x4 <3 
x3 +X%4 <1 
Mis wasn ear — Oior 


by a version of discrete improving search Algo- 
rithm 12C that always advances to the feasible neigh- 
bor with best objective value and uses the single- 
complement neighborhood permitting any one Me 
1 to be switched to = 0, or vice versa. 


(a) [] Identify a global optimal solution by inspec- 
tion. 

(b) &] Use Algorithm 12C to compute a local op- 
timum starting from x = (0, 0, 0, 0). 

(c) [Q] Apply the multistart extension of improving 
search to compute a local optimum by trying 
starts at x = (0,0,0,0), (0,1,0,0), and (0,0,0,1). 


12-46 Do Exercise 12-45 for the ILP 
min 50x; + 30x. + 20x3 + 15x4 
St. x +tx>1 
xX +xX4>1 
Ki, o-maarar a0 OFall 


Start Algorithm 12C atx = (1, 1, 1, 1), and multistart 
atx = (1,1,1, 1), (1,0, 1, 1), and (1, 1, 1, 0). 

12-47 Return to the improving search problem of 
Exercise 12-45. 


(a) Show that x = (1, 0, 0, 0) is a local optimum. 

(b) Show that if a nonimproving move is allowed at 
x = (1, 0, 0, 0), the next iteration will return the 
search to this same point. 


12-48 Do Exercise 12-47 for the model of Exercise 
12-46. 

12-49 [§] Return to the improving search problem of 
Exercise 12-45, starting fromx = (1, 0, 0,0). Com- 
pute an approximate optimum by tabu search Algo- 
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rithm 12D, forbidding complementation of a vari- 
able for one step after its value changes, and limiting 
the search to thax = 5 moves. 


12-50 Do Exercise 12-49 for the model of Exercise 
12-46. Forbid complementation of a variable for two 
steps after its value changes. 


12-51 [O] Return to the improving search problem of 
Exercise 12-45, starting from x) = (0, 0, 0, 1). Com- 
pute an approximate optimum by simulated anneal- 
ing Algorithm 12E, using a temperature of g = 20, 
limiting the search to tmax = 4 moves, and resolving 
probabilistic decisions with (uniform [0,1]) random 
numbers 0.65, 0.10, 0.40, 0.53, 0.33, 0.98, 0.88, 0.37. 


12-52 Do Exercise 12-51 for the model of Exer- 
cise 12-46. Use random numbers 0.60, 0.87, 0.77, 
0.43, 0.18, 0.13, 0.21, 0.48, 0.71, 0.83, 0.29. Start at 
x = (1,0, 0, 0). 

12-53 [§] Return to the improving search problem of 
Exercise 12-45. 


(a) [] Showthat the solutions x” = (0,0, 1,0) and 
x?) = (0, 0,0, 1) are eligible to belong to a ge- 
netic algorithm population for the problem. 

(b) [S| Construct all possible crossover results (all 
cut points) for the x and x® of part (a). 

(c) [| Determine whether all your resulting solu- 
tions in part (b) are feasible, and if not, explain 
what difficulty this presents for effective appli- 
cation of genetic algorithm search. 


12-54 Do Exercise 12-53 on the model of Exercise 
12-46 using x = (0, 1,1, 1) and x® = (1,0, 1,1). 

12-55 [| Return again to the model of Ex- 
ercise 12-45, and consider employing  ge- 
netic Algorithm 12F with initial population 
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{(0, 0, 1, 0), (0, 0, 0, 1), (0, 1, 1,0), 1, 0,0,0)}, p. = 
pi = 1, and p. = 2. Construct and evaluate eax 
member of the next generation population, we 
crossover after component 2 of the best and wor 
current solutions. Use a large negative M as th 
objective value of any infeasible solutions produce. 
by crossover. 
12-56 Do Exercise 12-55 on the mods 
of Exercise 12-46 with initial population 
{(0, 1,1, 1), 0, 0,1, 1), (0, 1,0, 1, Cd, 0, 0, 0)}. 
12-57 Consider solving (approximately) the follc 
ing knapsack problem by constructive search Alg 
rithm 12G. 

max 11x; + 1x. + 9x3+17x4 

8.t. Ox; + 2x2 + 7x3 + 13x4 < 17 

Xie ede = oni 


(a) [|] Determine a global optimum by inspectic= 
(b) [Q] Explain why it is reasonable to fix variab: 
in order of ratio 
objective coefficient 


constraint coefficient 
(c) E] Apply constructive search Algorithm 12 
to construct an approximate solution choosi=s 
variables to fix in this ratio sequence. 


12-58 Do Exercise 12-57 for the knapsack mode! 


min 80x, + 150x2 + 54x3 + 180x4 
st. 25x, + 30x2 + 18x3 + 45x4 > 40 
X1,...,X4 =Oorl 


Reeves, Colin R. (1993), Modern Heuristic Te 
niques for Combinatorial Problems, Halsted Pres 
New York: 
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Unconstrained 
Nonlinear Programming 


A major theme of this book has been the power and elegance of linear programming 
models, which are models with continuous decision variables, linear constraints, and 
a linear objective function. Nonlinear programming (NLP) encompasses all the rest 
of single-objective optimization over continuous decision variables. 

Being defined only by what it is not—linear—leaves nonlinear programming 
with a host of quite different forms and algorithms. Some models have constraints. 
Others have only an objective function. Calculus yields readily exploitable deriva- 
tives in many models, Derivatives do not even exist in others. In some cases, both 
objective function and constraints are nonlinear. In others it is only the objective 
‘unction. Even single-variable optimization is a nontrivial topic when the objective 
's nonlinear. 

This chapter begins our treatment of nonlinear programming with the uncon- 
strained case where no constraints apply. Chapter 14 follows with the more com- 
plicated models having constraints that cannot be ignored. There are important 
unconstrained applications, but most real models have at least a few constraints. We 
seat unconstrained cases first because many of the underlying notions of nonlinear 
programming are casier to understand without the encumbrance of constraints, and 
ecause most methods for constrained optimization use unconstrained algorithms 
ss building blocks. Familiarity with definitions of Section 2.4 and improving search 
soncepts of Chapter 3 is assumed throughout. 


13 ] UNCONSTRAINED NONLINEAR PROGRAMMING MODELS 
—/“e@ — 


Jne of the major differences between linear and nonlinear programs is that uncon- 
strained NLPs—ones with no constraints—can still be meaningful. 


2 begin our discussion of unconstrained NLPs with some typical examples. 


Tals) 
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EXAMPLE 13.1: USPS SINGLE VARIABLE 


Even models with a single decision variable can be challenging when the object*= 
function is nonlinear. For a real single-variable nonlinear program we turn to t= 
US. Postal Service (USPS).! Service “territories” for the USPS typically consist of « 
city and its suburbs. Mail delivery is provided by a number of postal carriers driving 
or sometimes walking, specified “delivery regions.” Carriers are based at “deliver 
units” distributed throughout the territory, beginning and ending their routes ther 
each workday. Often, a delivery unit is also a local post office that sells stamps. an- 
so on, to the general public. 

Determining the most efficient number of delivery units for a territory involy< 
a tradeoff between the fixed overhead costs of operating delivery units and the trav< 
savings from carriers being based nearer their delivery regions. More delivery un 
increases overhead, but it reduces the number of carriers required by saving tra‘< 
as the units are dispersed closer to customers. 

Increasing automation of mail handling has significantly changed the relat: 
economics of such decisions. To adjust, the USPS has developed and applied a rou 
decision model computing the approximate number of delivery units appropriate !~ 
any given territory. Input parameters are 


a A land area of the territory 


m A number of customers in the territory 

t A average time for a carrier to service any customer site 
d 4 length of the carrier work day 

c A annual cost per carrier 


u A annual overhead cost of operating a delivery unit 
We want to determine the decision variable 
x 4 number of delivery units 


To develop a model, we employ approximations derived from an assump? 
that customers are spread evenly over the territory. Under that assumption it can ™ 
shown that 


{ ; : a 
average travel time per carrier to/from regions ~ | 
EG 
travel time between route stops for all routes ~ k2./am 
where ky and k are constants of proportionality. Then the total number of routes 


(total time at stops) + (total time between stops) — #m + ky./am 
effective work time per carrier oh kya 


and total cost is 
tm+ a) 
d—k,./a/x 


overhead + operations = ux +c ( 


Based on D. B. Rosenfield, I. Engelstein, and D. Feigenbaum (1992), “An Application of Siz= 
Service Territories,’ European Journal of Operational Research, 63, 164-177. 


13.1 Unconstrained Nonlinear Programming Models WNT 


USPS Single- Variable Example Model 
To have a specific example with which to deal, pick 


a=400, m=200,000, d=8, t = 0.05 
c=0.10, u=0.75, ky 022 ky — 0.1 


in expression (13.1). Then our single-variable USPS nonlinear program is 
(0.05) (200,000) + no 
8) — (0.2),/400 
(8) — (0.2) /X (13.2) 


min f(x) A (0.75)x + (0.10) 


1089.4 


8 — 0.2,/400/x 


Figure 13.1 shows that a unique optimal solution occurs at x* ~ 15.3. 


& 0.75x + 


5) 10 15 20 25 30 35 
FIGURE 13.1 USPS Example Objective Function 


Neglecting Constraints to Use Unconstrained Methods 


Strictly speaking, model (13.2) is incomplete. A meaningful number of deliver units, 
x, should satisfy the constraint 


x>0 
and perhaps also 
x integer 


Still, we know (e.g., principle | 7.40 |) that adding a constraint can change an optimal 
solution only if it is violated. With x* = 15.2 > 0, the neglected nonnegativity con- 
straint would have no impact even if we modeled it explicitly. Integrality is violated 
at x*, but the rough planning nature of the model suggests that we would be quite 
justified in rounding to x = 15. 

Many, perhaps most, problems modeled as unconstrained nonlinear programs 
actually have a few constraints that are neglected in this way. 


es neglect- 
computed, 
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Of course, if the unconstrained optimum violates an important constraint, we mus? 
resort to constrained methods. 


Curve Fitting and Regression Problems 

Perhaps the most common of unconstrained NLPs involve curve fitting or regression 
We seek to choose coefficients for a functional form to make it fit closely some 
observed data. 


ion problem are the coefficients of the 


function measures the accuracy of 


13.3 | Decision variables in a teg 
fitted functional form, and the ob 
the fit. = SS 


EXAMPLE 13.2: CUSTOM COMPUTER CURVE FITTING 


For a simple example of curve fitting, we consider the problem of (fictitious) Custom 
Computer Company, which builds specialized computer workstations for engineers 
Although there may be many commonalties, stations produced for each order are 
specially modified to meet customer specifications. 

Table 13.1 shows the number of units and unit cost (in thousands of dollars 
of m = 12 recent orders. Figure 13.2 plots this experience. Obviously, unit cost 
declines dramatically with the size of an order. Custom wants to fit an estimating 
function like the one depicted in Figure 13.2 to facilitate preparation of bids on future 
work. 


TABLE 13.1 Cost Data for Custom Computer Example 


Number, Number, Number, Cost, 


Pi 


Cost, 


Cost, 


FIGURE 13.2 Fitted Curve for Custom 
Computer Example 


13.1 Unconstrained Nonlinear Programming Models 


Linear versus Nonlinear Regression 


Our first task in dealing with Custom Computer’s problem is choosing a regression 
form to fit. That is, we want to choose a function r(p) with unknown coefficients to 
approximate 
r(pi) © Gi foralli=1,...,m 

Nonlinear optimization will choose the best coefficient values. 

The fact that functional coefficients constitute the decision variables (principle 
133)) in curve fitting leads to considerable confusion. For example, regression 
analysts often denote a form something like 


re) 2 Foe (13.3) 


with x being data and {a, b} the coefficients to determine. This choice of notation is 
exactly the reverse of the one familiar in optimization. 

We will follow the mathematical programming tradition of reserving x for de- 
cision variables. Thus form (13.3) might be expressed as 


r(p) 2x1 + xp (13.4) 


with p the data and {x1, x2} the undetermined coefficients. 
A similar confusion arises in distinguishing linear versus nonlinear regression. 


roblem is termed linear ifthe function al form being fittedis — 
coefficients (decision variables) and nonlinear otherwise. _ 


For example, the choice of 
x 
r(p) Aart = (13.5) 


in our Custom Computer case would be a linear regression. The corresponding 
curve in Figure 13.2 would not be a straight line, but expression (13.5) is linear in the 
unknown coefficients x; and x2. 

The distinction between linear and nonlinear regression is important because 
there is often a closed-form solution for fitting linear forms, but nonlinear ones usually 
require search. Computations throughout this chapter will illustrate for nonlinear 
regression form 


r(p) 2 xp? (13.6) 
on our Custom Computer example. The curve depicted in Figure 13.2 is 
r(p) 4 40.69p~ 0-4 


which provides an optimal fit. 


Taking variables x; to be the unknown coefficients and all other symbols as given 
data, determine whether fitting each of the following forms is linear or non- 
linear regression. 


ee) 
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(a) r(p) 2 x1 + x2 sin(p) 
(b) r(p) 2 x1 + sin(@x2)p 
(c) r(p1, p2) A xipt + x26” 


Analysis: We apply definition : 


(a) This regression form is linear because it is linear in the decision variables x; am 
x2 for given data p. 


(b) This regression form is nonlinear. Decision variable x2 appears in the nonline= 
expression sin (x2). : 


(c) This regression form is linear. It fits a nonlinear function of two inputs p; and » 
but decision variables x; and x2 occur linearly. 


Regression Objective Functions 

To complete formulation of curve fitting as a nonlinear optimization, we require = 
objective function measuring fit. The error or residual associated with any data po= 
is the difference between the fitted function and the actually value observed. F 
example, in our Custom Computer case, residuals under functional form (13.6) ar 


gi — 1(pi) = Gi — x1 (pi) for alli=1,...,m 


Regression objective functions minimize some nondecreasing function of the mag= 
tudes of residuals. Many possibilities have been employed, but the most commo= 
the sum of residual squares or least squares. This objective possesses a number of ¢: 
sirable statistical properties, and it also has the intuitive appeal that small deviatio= 
cost little but large ones are heavily penalized. 


Custom Computer Curve Fitting Example Model 


For our Custom Computer example, the least squares objective produces unoo= 
strained NLP model 


m 


min f(x) 4 )° [qi —m(pi?) (137 


i=1 
Figure 13.3 displays the objective graphically, with global minimum at 
xj © 40.69, x5 ~ —0.6024 
yielding the best fit. 


RMULATING NONLINEAR REGRESSION Moves 


SAMPLE EXERCI 


Three heervaione fon a function believed to have the form 
Fr = a BY 
are (#4, Vi, Z1) = (1; 8, 3), 4, ¥2,.z3) = G, 15, 2), and (3, 3, 23) = 2,29) 7 te 


mulate an unconstrained nonlinear program to choose the a and £ yielding a lex 
squares fit. 


13.1. Unconstrained Nonlinear Programming Models 


F(%1, x2) (x}, x3) = (40.69, -0.6024) 


30 


a 45 


FIGURE 13.3 Objective Function in Custom Computer 
Example 


Modeling: Residuals are (z; — vB’). Thus a least squares fit will be obtained at a 
and £ solving 


min f(a, B) = (3 = a'p®): + (2 - apis): Elly eae | 


Maximum Likelihood Estimation Problems 


Another common application of unconstrained NLP arises in fitting continuous prob- 
ability distributions to observed data. A probability density function, d(p), charac- 
terizes any such distribution by showing how the probability is spread over different 
values of a random variable, P. For example, the d(p) depicted in Figure 13.4 indi- 
cates relatively higher probability of values of P near 0.7 than near 0.2. 


2.5 
2 
arms 
oy 
Se 
0.5 
0 a | 
0 2 = aa ae 


P 


FiGurE 13.4 Density Function for PERT 
Maximum Likelihood Example 


When say m independent random variables P;, Pz, ..., Pm have the same prob- 
ability density d(p), the joint probability density function or likelihood is 
d(p1, P2, +++, Pm) = d(p1)d(p2)---d(pm) (13.8) 


That is, the density for any particular independent combination of values is the prod- 
uct of the densities for the values separately. 
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Estimation involves choosing values for unknown coefficients of a density func. 
tional form d(p). The best estimates by many criteria are maximum likelihood on= 
that maximize the joint density of a known sample of random variable values. 


ation are the coefficients 
tion is the cores 
red data. 


joint density or lil lihood evaluated at the ob 


EXAMPLE 13.3: PERT MAXIMUM LIKELIHOOD 


Maximum likelihood estimates for parameters of many standard probability dens- 
ties can be obtained in closed form. However, some require a numerical optimizatios 
to maximize the likelihood function. 

A specific example occurs in fitting the beta distribution often used in project 
evaluation and review technique (PERT) project management. PERT is an extensio= 
of the CPM project scheduling method introduced in Section 9.7. As with CPM. = 
project is divided into a series of work activities with specified time durations. T= 
new element with PERT is that durations are taken as random variables (i.e., the 
are assumed to be known only in probability distribution at the time that plannins 
takes place). 

Beta random variables assign probability density over the interval 0 < p < 
so that in PERT they reflect the fraction p that some activity’s duration forms of == 
allowed maximum. The beta probability density function is 


d(p) & D(x + X2) 
~ PQ) 2) 


where x; > 0 and x2 > 0 are parameters controlling its shape.” For instance, x, = 
4.50, and x2 = 2.20 yields density shown in Figure 13.4. In expression (13.9), T(x 
the standard T’-function equal to the area under the curve 


y(x) A (hy te)" 


over 0 < h < +00. I(x) has no closed form. 

Table 13.2 shows the data we will assume for m = 10 previous times a projex 
activity was undertaken. Values p; represent actual duration as a fraction of t% 
maximum ever expected. 


(py a py?r? (138 


TABLE 13.2 Realized Data for PERT Maximum 
Likelihood Example 


Value 


Pi : P3 0.52 | Ds 0.74 | pz 0.79 | Dy 0.92 
P2 E Py 0.72 | pe 0.30 | pg 0.89 | pig 0.42 


Beta parameters are more commonly called « and £, but we will employ x; and x2 to be consist== 
with the convention that x; refers to a decision variable. 


13.1 Unconstrained Nonlinear Programming Models 


PERT Maximum Likelihood Example Model 
The beta probability density for observation p; = 0.65 is 
Px + x2) 
Px) Q2) 
and for the first two observations together it is [applying expression (13.8)] 


1g ; 
d(p1)d(p2) = Baeoomne = 0.65)" | 


; T@1 + x2) 
Nay) P@2) 


Continuing in this way our nonlinear program to maximize the likelihood of 
the full m-value sample over coefficients x; and x2 is 


PE Tr $22) : 
A F m1 jee f x—1 : 
max f(xy,x%2) 4 Wren? (1 — pi) (13.10) 


Figure 13.5 plots this objective for various x; and x2. The unique global maximum is 
at x} © 4.50 and x; ~ 2.20 of the density in Figure 13.4. 


d( pi) = d(0.65) = (165)7 AG — 0.65)7—+ 


04714 = osn| 


f(%1,%2) (xt, x5) = (4.50, 2.20) 


20 

15; AAR & 

10+ {/ Ren 
5 POSS 


TS SS>= + 
ed 


Figure 13.5 PERT Maximum Likelihood Example 
Objective Function 


SISE 1. TING MA IHOOD MODELS 
Exponential probability distributions have density function - ies, 

) d(p) £ ae~” 

Formulate an unconstrained nonlinear program to determine the maximum likeli- 

hood value of parameter a consistant with realizations p; = 4, po = 9, and p3 = 8. 


_ Modeling: The likelihood is the product [expression (13.8)] of the densities for the 
_ three realizations. Thus the model required is 


max f(a) 4 (ae **) (ae **) (we **) 


W123 
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Smooth versus Nonsmooth Functions and Derivatives 


It is useful to classify nonlinear programs according to whether their objective func- 
tions are smooth or nonsmooth. 


13.6 A function f(x) is sai be sm: th if it is continuous and differentiable 


at all relevant x. Otherw. 


Figure 13.6 illustrates this for some functions of a single x. (Refer, if needed, « 
Primer 2 in Section 3.3 for a quick review of differential calculus.) 


FQ) f(x) fx) 


wae 


(a) Smooth (b) Not continuous (c) Not differentiable 


I 


FIGURE 13.6 Examples of Smooth and Nonsmooth Functions 


The smooth/nonsmooth distinction is useful because the more erratic nature « 
nonsmooth functions usually implies a more difficult search. 


ie _13.7__ Nonlinear programs over smooth functions are ae more tractable 
_ than those over nonsmooth ones. 


SAMPLE EXERCISE 13.4: RECOGNIZING 


MOOTH FUNCTIONS 


Determine whether each of the following single variable functions is smooth or nos 
smooth over the specified domain. 


(a) fx) 4x? for x € (—od, +00) 
(b) f(x) 4 |x — 1| for x € (—oo, +00) 
(0) flx) 4 = for x > 0 


Analysis: We apply definition | 13.6 
(a) This function is differentiable at every point in the domain, so the function 
smooth. 


(b) This function is not differentiable at point x = 1, which is within the range. Thus 
the function is nonsmooth. 


(c) This function is discontinuous at x = 0. However, that point is not in the doma= 
specified. Thus the function is smooth for x > 0. 


13.1 Unconstrained Nonlinear Programming Models 


Usable Derivatives 


All three objective functions for the examples of this section (Figures 13.1, 13.3, and 
13.5) are smooth. Still, the existence of (partial) derivatives at every relevant x does 
not necessarily imply that derivatives are readily available to aid search algorithms. 


For the USPS example 
1089.4 
BIS, Pee EU 
IG) *T 80.2, /400/x 
and 
has 1089.4 ( 0.1 )(2) cena 
yo 2 2 ; 
ae (8—0.2/400/x) \v400/x] \* 


For the Custom Computer case 
pC se eo) = ys [gi — i (py? P 
with Ns 
Wd = ss [qi — x1(pi)? | (pi)? 
i=1 


Oxy 
of ao 
ats Yo [ai — 21") ba (i? | In@) 
*2 i=l 
Neither (13.11) nor (13.12) is a particularly simple expression, but both show deriva- 
tives that can be computed with reasonable effort to speed a search. 

In elementary calculus, unconstrained optima are computed by solving the sys- 
tem of equations resulting from setting derivatives = 0. With complex derivative ex- 
pressions such as (13.11) and (13.12), solving such a system of equations is often as dif- 
ficult as solving the underlying nonlinear program. Nevertheless, practically available 
derivatives can be a significant aid to a numerical search for optimal variable values. 


(13.12) 


deri atives — 


Contrast with the PERT maximum likelihood objective 


— Fe + X2) 


A \x1-1 — ».)X2-1 
fo.) ST] Papros) ? (1—pi 


The [’-function itself has no closed form, so derivatives are certainly not readily avail- 
able, even though they do exist in theory. To compute an optimum for this case we 
require a search method that does not depend on derivatives. 


d25 
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min fla, B) A (3- a'p*): + (2- apis). + (71 — op)" 


(a) Express partial derivatives with respect to a and £. 


(b) Discuss the usefulness of those partial derivatives in computing an optimal o ans 
p. 

Analysis: 

(a) Partial derivatives are 


of 
oa 


= 2(3 — a! B*)(—p%) + 2(2 — a4 BS) (—4a3 p!5) 
+2(71 — a B”?)(—206??) 

z = 2(3 — a! B§)(—8a' B”) + 2(2 — a4!) (-1504 8!) 

+2(71 — a p*?)(—29a pS) 


(b) Although derivative expressions of part (a) are somewhat complicated, they c= 
be evaluated efficiently. Simply setting them = 0 is impractical because it wous 
leave a pair of difficult nonlinear equations to solve. However, the derivatives cas 


38). 


be employed to speed an improving search (principle 


13.2 ONE-DIMENSIONAL SEARCH 


The easiest case on unconstrained nonlinear programming is single-variable or & 
dimensional search. One-dimensional NLPs occur both directly, as in our USPS 
example of Section 13.1, and as line search subroutines choosing step sizes to apr 
to move directions of more general algorithms. 


Unimodal Objective Functions 
Figure 13.7 illustrates for 
fe) £@-4@- 6° - 1) 
how 1-dimensional optimization can be quite challenging. 
e Points x® and x® are both local minima because small changes from either x do not 
crease the objective function (definition | 3.5 |). Only x is an overall, global minimax 


(definition ) for the displayed interval. 


e Point x® is a local maximum because small changes from x do not increase the > 
jective function. Being the only local maximum, it is also a global maximum for © 
displayed interval. 


e Point x is neither a maximum nor a minimum, despite the fact that the slope of * 
(derivative df/dx) at the point = 0. 


Fortunately, most of the 1-dimensional searches we encounter in applicatioe 
are somewhat better behaved. Recall from definition in Section 3.4 that = 


tO 
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150 


100 


50 


(1) x) x@) x) 


FIGURE 13.7 Single-Variable Nonlinear Function 


objective function f(x) is unimodal if the straight-line direction Ax to any x with 
a better objective value is an improving direction; that is, it helps the objective for 


even very small positive steps (definition ). Over unimodal objective functions, 


every unconstrained local optimum is a global optimum (property ). 

The “one-hump” or “single-mode” character that gives unimodal functions 
their name is particularly easy to grasp in 1-dimensional cases. For example, the 
minimizing USPS objective of Figure 13.1 is unimodal because the objective is de- 
creasing (improving) at every x to the left of x* ~ 15.2 and increasing (degrading) 
at every x to the right. Since a tiny move toward x* always helps, the local minimum 
there must be global. 

Notice that an objective’s being unimodal depends on whether we are maximiz- 
ing or minimizing. The USPS objective of Figure 13.1 is not unimodal for a maximize 
problem because, for example, a move toward x® = 30 does not immediately im- 
prove the objective function at x“ = 10, even though f(30) > (10). 


Golden Section Search 


Although derivatives can sometimes be of assistance, many 1-dimensional optimiza- 
tions employ simpler methods not requiring derivatives. Among the most clever is 
golden section search, which deals with an unimodal objective by rapidly narrowing 
an interval guaranteed to contain an optimum. 

Figure 13.8 illustrates the idea for a minimize problem. We iteratively consider 
the functional value at four carefully spaced points. Leftmost x" is always a lower 
bound on the optimal x*, and x is an upper bound, so that an optimum is certain 
to lie within the interval [x°, x]. Points x® and x fall in between. 

Each iteration begins by determining whether the objective is better at x or 
x, If x proves superior [part (a)], we may conclude the optimum lies within the 
smaller interval [x", x]. If x has a better objective value [part (b)], we restrict 
further attention to the interval [x®, x]. 
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f(x) Ff) 
A A 


| 


I pel ees eae 1 Hs 
lo) oe wi rid * lo) 7 30.11 «(pom 
l ! ! } | I I 
x(lo) xG) x2) x(hi) x(lo) (1) (2) (hi) 
(a) Optimum left (b) Optimum right 


| | | 


FIGURE 13.8 Interval Reduction in a Minimizing Golden Section Search 


Each narrowing of the interval certain to contain an optimum leaves us with 2 
endpoints and 1 interior point at which we already know the objective value. The 
efficiency of golden section search comes from how we choose 1 new interior point te 
evaluate. Any new point would allow the search to continue, but we want consistent 
progress regardless of whether the next interval is [x", x] or [x®, x9], 

Golden section search proceeds by keeping both these possible intervals equa’ 
in length. 


ae 10. aul two middle points of golden section search are cree E 
ee ee 


— oe xil0) = (xd — oO) = — “ ee 


oe aX 0). 618i isa a fraction known as the golden ratio. — 


Whichever of [x°, 2 or [x“), x0] provides the next interval, its length will be « 
times the current. 
The golden ratio value of 


_ -14+V75 
i 2 
arises from the need to maintain the spacing of principle | 13.10/ as the algorithm 


proceeds. For example, suppose that the chosen next interval is [x", x]. As 
indicated in Figure 13.8(a), we want current x™ to play the role of x® in the next 


~ 0,618 (13.13 


interval. Applying formulas | 13.10 | yields 
current x) = x®) — gd _ yo) 
and 
next x? = x") 4 q(x — x) 
= x6) 4 gy (x00) $ gett — x0) — C00) 
Equating and regrouping produces 


oS a? (xd sy 0) si ar(x) e x0)) ut (acd) “3 x0) 
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which further simplifies with x? 4 x" to 

0=e°+a-1 
The unique positive root of this quadratic equation is w = the golden ratio of expres- 
sion (13.13). 


Golden Section Solution of USPS Example 


Algorithm 13A formalizes these ideas of golden section search. Table 13.3 details its 
application to our minimizing USPS model (13.2). 


ALGORITHM I3A: GOLDEN SECTION SEARCH 


Step 0: Initialization. Choose lower bound x“ and upper bound x“? on 
an optimal solution x* along with stopping tolerance « > 0, compute 


Dg thi) — yeti) _ 5-0) 


for golden ratio « of (13.13), evaluate objective function f(x) at all four 
points, and initialize iteration counter ¢ < 0. 

Step 1: Stopping. If (x — x") < €, stop and report as an approximate 
optimal solution 


xe 5 (x) + x{hi)y 


the midpoint of the remaining interval. Otherwise, proceed to Step 2 if f(x“) 
is superior to f(x) (less for a minimize model, greater for a maximize), and 
to Step 3 if it is not. 
Step 2: Left. Narrow the search to the left part of the interval by updating 
tt 
Oa a 
xD xh) _ oe (xh) = x0)» 


and evaluate the objective at new point x. Then advance t <—t+1, and 
return to Step 1. 
Step 3: Right. Narrow the search to the right part of the interval by updating 
x yO 
LD VO) 
x2) < x0) 4 ey(ethi) _ (lo) 


and evaluate the objective at new point x. Then advance ¢ <—¢+ 1, and 
return to Step 1. 


Computation in Table 13.3 starts arbitrarily with interval 
= [8.32 
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TABLE 13.3. Golden Section Search of USPS Example 


t xo) x) x) xh) fF) Fe) fe) F(e™) hi) _ (lo) 


0 8.00 17.17 22°83 32:00° "171.42" 167k 74 169.22 © “17338 24.00 
iL 8.00 13.67 17.17 22.83 171.42 167.73 167.74 169.22 14.83 
2 8.00 11.50 13.67 17.17 171.42 16836 167.73 167.74 ONG 
3 11.50 13.67 15.00 7.17... A68S6peqlo773i. 1675628 167 74 5.67 
4 13.67 15.00 15.83 17.17 167.73 167.62 167.63 167.74 3.50 
5 13.67 1449 15.00 15.83 167.73 167.64 167.62 167.63 2.16 
6 1449 15.00 15.32 15.83 167.64 167.62 167.61 167.63 1.34 
7 1500 15.32 15.51 15.83 ° 167.62 167:6lr ~ 16762>~ 167763 0.83 
8 15.00 15.209)95:32) 15.51 167.62 67-6 ool eemloTe2 0.51 
9) F1Si20) 15320 S298 155i” M1G7:61 ) 16 7-oie ile ole67462) 0.32 


It is only necessary that the interval contain the optimum. Intermediate points x ° 
and x are then computed by principle as 
xD e zxhi) a(x") — x0) 
= 32 — 0.618(32 — 8) 
17.17 
x < xflo) 4 x(x) — x(0)) 
= 8 + 0.618(32 — 8) 
= 22.83 
At iteration ¢ = 0, objective value 


fixe) = 167.74 < f(x) = 169.22 


implying that the optimum lies in the left part of the current interval. Following 
Algorithm 13A, we update 


xO) < x@ = 22.83 
x <— xO = 17.17 
and compute new 
xD < xh) — qy(y(hi) _ xo) 
= 22.83 — 0.618(22.83 — 8.00) 
~~ 13.67 


The process now repeats for iteration t = 1. 
Computation continues until interval [x°), x] has sufficiently small length 
In Table 13.3 stopping was set to occur when 


x) _ yf) - € = 0.5 
which happened at t = 9. Then our estimate of the optimal solution is midpoint 
x <— F(x 4+ x) = 515.20 + 15.51) = 15.36 


If greater accuracy were desired, we would need to continue through more iterations 
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nonlinear program 


(x — 20)* 


A 22 
max. fG) = 2% 500 


plotted below. 


Continue until the interval containing an optimum has length at most 10. 


Analysis: The algorithm proceeds exactly as in Table 13.3 except that this model 
maximizes. The following table provides details: 


ES ee ee Meee: a eas 
txt) DQ gli) f(y) FD) FX) forth thi _ (0) 
a a ee 


Q 0.00 15.28 24.72 40.00 -320.00 29.57 48.45  -240.00 40.00 
1 15.28 24.72 30.54 40.00 29.57 48.45 36.27. -240.00 24.72 
Oe 1s28 maPll2 2472 30:56 29.57 42.23 48.45 36.27 15.28 
Boe 2d Din 24 1Dew26.95~ ~ 30,56 42.23 48.45 49.23 36.27 9.44 


Termination occurs at f = 3 with x) — x) = 9.44 <e€=10. 


Bracketing and 3-Point Patterns 


Golden section search begins solving a 1-variable model with an interval [x", x] 
known to contain an optimum. But how do we determine such initial intervals; that 
is, how do we bracket an optimal solution before the main search begins? 

Sometimes the initial bracket is given because the model includes implicit upper 
and lower bounds on the decision variable. Much more commonly, one endpoint is 
known and the other must be determined. For example, in line searches, where 
the single variable is the step size 4 to apply to a chosen move direction, 2 must be 
positive. Thus we begin with x" = 0. 

To locate the corresponding x" bracketing the optimum of a unimodal objec- 
tive function requires a search for a 3-point pattern. 


Veal 
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13.11 In 1-dimensional optimization, a 3-point pattern is a collection o: 
sion variable values x" < x‘ < +) with the objective at x) supe- 
rior to that of the other two , lesser for a minimize). 


Figure 13.9 illustrates for our USPS model (13.2). Points 
x99) 8 P= 16; ax Me 
surround the minimum in a 3-point pattern with 
fx) = f(8) © 171.42 > fix™®) = f16) © 167.63 
fx) = fB2) © 173.38 > fix) = f(16) © 167.63 


280 
260 
240 
220 
200 


180 


160 
eal yo a8 xlmid) — 16 res) 


FIGURE 13.9 Bracketing the USPS Optimum with a 3-Point Pattern 


A 3-point pattern provides the bracket we seek if the objective function is uni- 
modal. Midpoint value f(x) superior to f(x") means that the function improves 
to the right of x), Similarly, with f(x) better than f(x”), the function improves 
to the left of x. An optimum must lie in between. 


13.12 | Tf (x0), ximid 


f(x), there is an opti: 


Finding a 3-Point Pattern 


Algorithm 13B details the most common scheme for quickly finding a 3-point patter= 
when we are given only an initial lower endpoint x“. Values of x are modified by 
exponentially changing step 6 until the last three form a 3-point pattern. 


ALGORITHM I3B:THREE-POINT PATTERN 


Step 0: Initialization. Choose lower bound x“ on optimal solution x* and 
initial step 5 > 0. 

Step 1: Right or Left. If f(x" +8) is superior to f(x) (less for a minimize, 

greater for a maximize}, set 


(mid) <— xo) a 5 


and go to Step 2 to search right. Otherwise, an optimum lies to the left; set 


hi) <— xo) +6 


and go to Step 3. 
Step 2: Expand. Increase 5 < 28. If now f(x") is superior to f(x™ + 6), 
set 


(hi) < x (mid) a5 


and stop; {x, x x(4)}) forms a 3-point pattern. Otherwise, update 


x (lo) vis x mid) 


x mid) <— (mid) a5 


and repeat Step 2. 
Step 3: Reduce. Decrease 5 < $3. If f(x" + 8) is now superior to fix"), 
set 


x fmid) <— xo) a 


and stop; {x «(mid )} forms a 3-point pattern. Otherwise, update 


xd) — 00) 4 g 


and repeat Step 3. 


Values in Figure 13.9 illustrate the idea. Computation starts with x"? = 6 = 1. 
Since 


fx +8) =f 41) = f(2) © 212.16 < f(x) = fd) » 273.11 


we must expand to the right to find a 3-point pattern bracketing the optimum. Setting 
x(mid) — 2 we double 6 and consider 


xl) + §=242=4 
The function improves again, so 

lO ey id ak 9 

xmid) <— y(mid) + §— 4 


and the process continues. 
Eventually, we have x" = 8, x) — 16, and 6 = 16. Then 


fc) = f16) © 167.63 < fix + 6) = f(32) ~ 173.63 
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and the algorithm stops after completing the 3-point pattern with 
xO) — xrid 1 5 = 32 


| SAMPLE EXERCISE I. 


Return to the model 


(x — 20)* 
500 


of Sample Exercise 13.6, and apply Algorithm 13B to compute 3-point patterns wit® 
initial x" = 0 and (a) 5 = 10; (b) 6 = 50. 


max f(x) 42x — 


Analysis: 


(a) Initial fe") = f() = —320, and fix +8) = fa0) = 0 improves. Thus 
x™) <— 10. Doubling 6 and trying f(x™ + 5) = f(30) = 40 produces furthe: 
improvement. Thus x") < x(mid = 10, xd ~< 30. Doubling 6 again yields 
fe + 8) = f(30 + 40) = -12,360. Thus we stop with x“) — 70, 


(b) Initial fox") = f(0) = —320, and f(x + 8) = f(50) = —1520 is worse. Setting 
x‘) < 50, we could stop if our only purpose is to bracket the maximum. To complete 
a 3-point pattern, however, we must reduce 5. Halving to 5 = 25 produces f(x") +8) = 
f(25) = 48.75, which does improve on f(x"). Choosing x") — 25 completes the 
3-point pattern. 


Quadratic Fit Search 


Golden search Algorithm 13A is reliable, but its slow and steady narrowing of the 
optimum-containing interval can require considerable computation before an Opti- 
mum is identified with sufficient accuracy. Quadratic fit search closes in much more 
rapidly by taking full advantage of a current 3-point pattern. 

Given a 3-point pattern, we can fit a quadratic function through corresponding 
functional values that has a unique maximum or minimum, x(@ , whichever we are 
seeking for the given objective f(x). Quadratic fit uses this approximation to improve 
the current 3-point pattern by replacing one of its points with approximate optimum 
x(q) 

Figure 13.10 illustrates for USPS model (13.2) with initial 3-point pattern 

sa MEE sae Bo TS ade) 


The main curve plots actual objective function f(x). A second, dashed line shows the 
unique quadratic function fitting through the 3 pattern points. 
That quadratic approximation has a minimum at 


x ~ 18.56 with fo) ~ 167.98 (13.14 
Together with the current x“ and x" it now forms a new 3-point pattern 
xo 8, gl = 1856.0 7 = 20 


with a smaller interval [8, 20]. Repeating in this way isolates an optimum for f(x) iz 
an ever-narrowing range. 
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fitted 


quadratic 
x 
~ x 


~ 


x(lo) = g x(@) =18,56 (mid) — 99 xthi) = 32 
FIGURE 13.10 Quadratic Fit Search of USPS Example 


The computation to determine x” requires only some tedious algebra. 


For example, the approximate minimum of expression (13.14) is 


1171.42 [ (20)? — (32)?] + 168.32 [(32)? — (8)?] + 173.38 [ (8)? — (20)?] 
2 171.42 [20 — 32] + 168.32 [32 — 8] + 173.38 [8 — 20] 
18.56 


Quadratic Fit Solution of USPS Example 


Algorithm 13C details a quadratic fit procedure for 1-dimensional search, and 
Table 13.4 tracks progress for our USPS example. The reader can verify that each 
iteration produces a new 3-point pattern and that interval [x“°, x] narrows con- 
stantly. Computation for Table 13.4 was stopped when that interval had length at 
most € = 0.50. 

One new element arises when the computed x“™ happens to nearly coincide 
with the current x. If nothing were done, the algorithm would loop forever. Step 
3 of Algorithm 13C addresses this difficulty by preturbing x“ by €/2 toward the 
most distant endpoint. 

The one value changed in this way in Table 13.4 is marked with an asterisk 


(*) att = 7. There formula | 13.13 | produced x“ = 15.34, which was too close to 


xq) — 
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Step 0: Initialization. Choose starting 3-point pattern {x"), x, xD} glong 
with a stopping tolerance € > 0, and initialize iteration counter t < 0. 

Step 1: Stopping. If (x — x) < «, stop and report approximate optimal 
solution x™4 | 

Step 2: Quadratic Fit. Compute quadratic fit optimum x according to 


Step 4; and if x™ > x4 go to Step 5. 
Step 3: Coincide. New x coincides essentially with current x0, [f x(mi¢ 
if farther from x@° than from x‘, preturb left 
(mid) _ € 
2 
and proceed to Step 4. Otherwise, adjust right 


x 2 


xf < x(mid) - 


and proceed to Step 5. 
Step 4: Left. If f°) is superior to f(x) (less for a minimize, greater for 
a maximize), then update 
xf) x(a) 
Otherwise, replace 
x(n) (mid) 
ximid) ._ y(qu) 
Either way, advance t < t+ 1, and return to Step 1. 
Step 5: Right. If f(x‘) is superior to fix) (less for a minimize, greater 
for a maximize), then update 
xh) 2 Qu) 
Otherwise, replace 
xo) <x (mid) 


(mid). (qu) 


Either way advance t < t +1, and return to Step 1. 


TABLE 13.4 Quadratic Fit Solution of USPS Example 


™ 


xo) xfimid) hi Fe) Fea) F(x) x (hi) = xo) x(a) fx) 


O 8.00 20.00 32.00 171.42 168.32 173.38 24.00 18.56 167.98 
1 8.00 1856 20.00 171.42 167.98 168.32 12.00 16.75 167.70 
Din 8.00,. 16:75 ©) 185605 M7142 167.70 167.98 10.56 16.23 167.65 
3) 8.00 - 16.23) +, 16:75 .* 171.42 167.65 167.70 8.75 15.78 167.62 
4 8.00 15.78 16.23 171.42 167.62 167.65 8.23 15.59 167.62 
a 800 1559), 1578) ea? 167.62 167.62 7.78 15.45 167.62 
6) 98100 oi 15 4559155998 171042, 167.62 167.62 7.59 15.38 167.61 
Tg S008 lS. 38-4 15 AS) lea: 167.61 167.62 7.45 ATS 135, 16762 
SPelSalst = (15538) Fas 45S S16 762 167.61 167.62 0.32 — — 
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xm) — 15.38. With x) = 8.00 farther from this value than x) = 15.45, Step 3 
preturbed the computed value to 


x) = mid) _ 2295198 40.95 = 1543 


Return to the unconstrained nonlinear program of Sample Exercise 13.6: 
(~ — 20)* 

500 
Using initial 3-point pattern x" = 0, x) = 32, x%) — 40, apply quadratic fit 
Algorithm 13C to identify an optimal solution within an interval [x x] of length 
at most 10. 


max f(x) 42x—- 


Analysis: Computation parallels Table 13.4 except that this model maximizes. De- 
tails are contained in the following table: 


tx) gltmidy tht) Fpl) f(eCmid) y(t) phi) _ (lo) qu) F(y-(4)) 


0 0.00 32.00 40.00 -320.00 22.53 -240.00 40.00 20.92 41.84 
1 0.00 20.92 32.00 -320.00 41.84 22.53 32.00 25.00 48.75 
2, 20.925 25100) 4.32.00 41.84 48.75 22.53 11.80 *30.00 40.03 
3.4 2092 30.00 41.84 48.75 40.03 9.08 — — 


13.3 DERIVATIVES, TAYLOR SERIES, AND CONDITIONS 
FOR LOCAL OPTIMA 


Unconstrained nonlinear optimization is certainly possible without derivatives. Still, 
where derivatives are readily available, they can tell us a great deal about a model 


and substantially accelerate search algorithm progress (principle| 13,8 |). This section 
develops some of the most important insights to be gained. 


Improving Search Paradigm 


In Sections 3.1 and 3.2 we introduced the principle of improving search (Algorithm 
3A, Section 3.2) on which almost all nonlinear algorithms are based. We begin with 
a (vector) solution x satisfying all model constraints. In the unconstrained context 
of this chapter, any x will do. Iterations ¢ advance current solution x to 


xD <— x 4 Ax 
where Ax is a move direction and A a positive step size. Each Ax should be an improv- 
ing direction; that is, it should produce immediate objective function improvement 


(definition ). (It should also retain feasibility when constraints are present.) 
The process continues until a point is reached where no directions lead to such im- 


mediate improvement. There we stop (principle | 3.17 |) with what is usually a local 


optimum—a point as good in objective value as any nearby (definition | 3.5|). [See 


Figure 3.8(a) for an exception that is not a local optimum even though it admits no 
improving direction.] 
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Figure 13.11 illustrates for our (minimizing) Custom Computers model (13.7 
At initial point x = (32, —0.4) that search took a step of A = 4 in direction Ax = 
(2, —0.2) to produce 


32 2 33 
ine 1 iy 
=( ae 


Dashed lines in the figure, which show contours of the objective function plotted = 
Figure 13.3, demonstrate that the move is improving. Even very small steps from x 
in direction Ax advance the search to lower contours of the objective function. 


ra 

i @ x) = (33, 
\ 3 / 

‘ \ 


N 


*4 


FIGURE 13.11 Improving Search of the Custom 
Computers Example 


Local Information and Neighborhoods 


What move direction should the search of Figure 13.11 adopt next? The best choic: 
would make 


: 40.69 33 7.69 
eet (1) = a 
Ti ea ( ~0.6024 ) ( ~0.5 ) ( ~0.1024 ) 
which leads directly to the optimal solution. 
Unfortunately, a search in progress does not have the global viewpoint availa> < 
in Figure 13.11. The next move must be chosen using only experience with poin™ 


already visited (here x and x“) plus local information about the shape of th: 
objective function in the immediate neighborhood (definition [3.4]) of current x 


First Derivatives and Gradients 


We know from elementary calculus (see also Section 3.3 and Primer 2) that firs 
derivatives or gradients provide information about how an objective function changes 
near a current solution x, 


, of/0x, with m 
increments in 


13.3. Derivatives, Taylor Series, and Conditions for Local Optima 


For instance, at x = (33, —0.5) in minimizing Figure 13.11, we may apply 
partial derivative expressions (13.12) to compute 


a ) 
f ~~ —23.07, yf ~ —174.23, sothat Vx) ~ (—23.07, -174.23) 
Oxy Ox2 


Thus small increments from either x; = 33 or x2 = —0.5 decrease f(x1, x2), but the 
rate of change is much more rapid with increments in x). 


Second Derivatives and Hessian Matrices 


When an objective function is twice differentiable, which is typical for the smooth 
objectives most often occurring in applications, second derivatives can tell us still 
more about the shape of the function in the neighborhood of current solution x. 
Primer 7 reviews some of the fundamentals. 
The second derivative of a single-variable objective f is a scalar function f” (x). 

For an n-variable objective, there is a whole Hessian matrix of second partial deriva- 
tives with row i, column j entry 0°f/dx; dx;.. For example, our Custom Computer 
objective [model (13.7)] 

m 

fei) = oe G= xp”)? 

i=1 

has first partial derivatives [expression (13.12)] 


0 3 ; 
a =-2 2 (gi — x1p;°) (ip;?) n@—a 


Thus second a are 


2. 
— =290 ‘ 


ef ef 


Oxy OX2 0X2 Oxy 


% (13.15) 
=) Ds [ai — mp?) (P?) In@) — @) Gp”) In@~d] 


ie 


Ox 


23? (pi) [@i — 1p?) Cap?) — Cap?’ 


At the x = (33, —0.5) of Figure 13.11 constants p; and q; of Table 13.1 yield the 
Hessian matrix 

af of 

ax? xy xy | ( Sf 217965 ) 


OX 0X1 0x22 


H(33, —0.5) 4 
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Primer 2 (Section 3.3) provides a brief overview of first « rivatives d fildx [or 
f(a)| and first partial derivatives a f/ox; ‘B which measure the rate of change 
function f with respect to increases in its arguments. The vector of partial 


derivatives for n-variable function fx) 
First derivatives or partial derivati 

tions of its arguments. If such deri 

is said to be twice aificrem lable: a 


We 


2). RS 
Sécond derivatives of a singe iable f(x) 
or fo. For example, fix) A 3x7 erivative f'(x) = 12x* and second 
f'() = 36x7. At x fidx = 122) = 96, while d? f/dx* = 36(2)° = 144. 
‘Twice differe functions f(x) A fiq,..., x, ) of n variables have sec- 
s for each pair of variables x; and x;. Such second ee 
derivatives are customarily denoted & f/ax; ax; jwhen i 4 j and 0° f/ax;* if i = > 


he order i in which variables are listed tadicaies the sequence = differentiation. 


ef, 3 aise 
Ox: 0x = ax; 


To illustrate, consider f(x), x2) A 5x1 (20)°. First partial derivatives are 
0 f/ax, = 50x)? and 3 a = 15x10). Thus 


Notice in this — that a Feet = = 0° f/ax. It is always true that 


ae = a . 


when f and all its first partial derivatives are continuous functio 
Ati is often convenient to deal with s d partial derivatives in a Hessian 


Ox} Xy 


| af . = a7 
OX, Oxy . On? Ls 


ea O- ag 0 2 
15(2) > 60 —180 


What second derivatives offer a search algorithm is information about the == 
vature of objective function f near current solution x”. 


For example, the 0°f/dx2* = 11,003.12 confirms what we can see in Figure 13.11—that 
small changes in x2 dramatically affect the slope of f near x“. The much smaller 
0° f/9x1? = 5.77 indicates the function is flatter in the x, dimension. 


Taylor Series Approximations with One Variable 


A more concise description of what derivatives tell us about an objective function 
follows from classic Taylor series. For 1-dimensional function f(x), Taylor’s approx- 
imation represents the impact of a pe i from current x as 


FO +2) © fle) + HF) +5 + FG) + + FG) oti (13.16) 


where f"(x) is the first derivative of f, ee is the ne derivative, and so on. 
To illustrate, consider 
fase (13.17) 


for which f’(x) = 3e*-®, f’(x) = 9e**-®, and f(x) = 27e3*-§. Near x® = 2, 
derivatives approximate the impact of a ee Aas 


f2+d) * f2)+ =f) + + FQ) + * #0) +- 


=14+30+4 3224 234... 

Notice that as |A| + 0, higher powers of 4 approach zero the most rapidly. 
That is why we may approximate a function with just the first few terms of expression 
(13.16) if our interest centers on the immediate neighborhood of current x. The 
results are the first-order or linear, and second-order or quadratic approximation to 
a function of a single variable. 


quadratic Taylor series ap- : : 


= «© are, respectively, 


Figure 13.12 illustrates for the f(x) 4 e*~° of expression (13.17). Part (a) plots 
f(x) and the first-order approximation for current x = 2 


fi(2+d) =fQ) +Af'(2) =14+3a 
and part (b) shows f(x) versus second-order approximation 
fo(2 +4) = fQ) + Af 2) + 5°f"(2) = 14 3a + 3a? 


13.3 Derivatives, Taylor Series, and Conditions for Local Optima 
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(a) First order (b) Second order 


FIGURE 13.12 First- and Second-Order Taylor Series Approximations 


Notice that the first-order approximation is a linear function of change i. It 
assumes that the slope at x = 2 remains constant. Both approximations are fairly 
accurate near 4 = 0 and deteriorate as i becomes larger. Still, the second-order 
approximation in part (b) comes somewhat closer to the real f because it incorporates 
the curvature information in second derivative f’ (x). 


Taylor Series Approximations with Multiple Variables 


We may extend Taylor series approximations to functions of more than one variable 
by using first and second partial derivatives. 


__proximations ton 


As with 1-dimensional series (13.16), there are higher-order terms in the full Taylor 
series expansion of an -variate function, but they become insignificant as |A| > 0. 


13.3 Derivatives, Taylor Series, and Conditions for Local Optima 


To illustrate | , consider 


f(x1, x2) A x1 Ina) +2 
at x = (—3, 1). There f(—3, 1) = 2, and gradient 


a In(x2) 
Shae inh 

Vi(—3, 1) 4 - z M1 VAL) 
ax “ 


Thus the first-order approximation to f(x1, x2) near x = (—3, 1) in direction Ax 4 
(Ax;, Ax2) Is 


fix +AAx) A fx) +00 f%)- Ax 
=2+2(0, —3)- (Ax, Ax2) 


=2-3\A Ax 
To improve the approximation with second-order terms, we compute Hessian 
1 
Lee, 01 
H(-3, 1) = 7 oab= 
\ ) ale X41 ( Hee) ) 
x (x2)? 


Then 
$F 
fo(x® + AAx) 4 fx) 4A Vf) Ax + Zax H(x®)Ax 


A 2 0 1 A 
= 24400, -3)( ren nace axy( 4 an oa | 


= 2 —3A Ax, +)? Ax; Axy + 342(Axy)? 


Stationary Points and Local Optima 


First and second derivatives tell us a great deal about whether a solution is a local 
optimum. Begin with stationary points. 


That is, stationary points are solutions where all first (partial) derivatives equal zero. 
Figure 13.13 illustrates for 


fea, x2) 4 40 + GP — 4) + 32 — 5)? (13.18) 
Partial derivatives are 


a 

= = (a1)" (4x1 — 12) 

; : (13.19) 
ee) 

0x2 


It is easy to check that they become zero at two stationary points: 
x = (3,5) and x® = (0,5) (13.20) 


743 


744 Chapter Thirteen e Unconstrained Nonlinear Programming 


Ss 


FIGURE 13.13 Stationary Points of a Minimize 
Objective 


We can see in Figure 13.13 that one of these, x, is a local (and here also global 
minimum of f. This suggests our first (so called first-order necessary) condition for 
an unconstrained local optima. 


be a stationary point. 


The reason that condition | 13.19} must hold in every case is that a nonzer: 
gradient V/(x) itself provides an improving direction at x. Following principl: 


|3.23|, we may adopt Ax = +V/f(x) with + for maximize problems and — + 


minimize. Then the first-order Taylor series approximation eee: 


gives 
fx 4+ 2Ax) © f(x) + AVA) -Ax 
= ftx®) + AVA) 4 Vix) 


n af 2 
= f(x) 4A (=) 
f dX Ox; 


This is an improvement in the objective value unless all partial derivatives = 0, anc 
we know that for A sufficiently small the first-order part of the Taylor series expansio= 
dominates all other terms. 


(13.21 


Consider the single-variable function 


fx) =x? — 9x? + 24x — 14 


Plot the function for 1 < x < 5 and verify that local maximum x = 2 and locs 
minimum x = 4 are both stationary points. 
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Analysis: A plot of the function is as follows: 


=o 
tad 
~~ 


local maximum x“ = 2 


oF NY WH FN DBD YI 


D3 BS) Ay FASS 


Its first derivative is 
f GH 3x 18x +24 


Both f’(x®) = f’(2) = Oand f’(x™) = f'(4) =0, confirming that both are stationary 
points. 


Saddle Points 


Look again at Figure 13.13. Stationary point x = (3,5) is a local minimum, but 
x) = (0, 5) is not. Increasing x; reduces the objective at the latter. Point x® is also 
not a local maximum. Increasing x2 makes the objective value bigger. Figure 13.14 
shows that the remaining possibility is a saddle point. 


Every stationary point is either a local maximum, a local minimum, or a saddle point. 

Saddle points get their name from the saddlelike possibility of the 2-dimensional 
case in Figure 13.14(c). The same stationary point is a local maximum in one di- 
mension and a local minimum in another, yet neither a local maximum nor a local 
minimum when both directions are considered together. 


Hessian Matrices and Local Optima 


To distinguish better among the 3 types of stationary points in Figure 13.14 we must 
look at second (partial) derivatives. At stationary points, which have Vf(x) = 0, 


second-order Taylor approximation | 13.17 | simplifies as 


2 
f(x + Ax) © f(x) FAV F(R) Ax + as H(x)Ax 
2 (13.22) 


2 
= f(x) +04 Ax Hx )Ax 
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F (24; ¥2) F (x4, X2) 
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(a) Local maximum (b) Local minimum 


f(%1, *2) 


(c) Saddle point 


FiGurE 13.14 Three Forms of Stationary Points 


Thus (nonzero) Hessian-based quadratic forms AxH(x" )Ax critically influenc: 
whether improving directions Ax exist at stationary points (i.e., whether such poin= 
have any chance of being local optima). 

Consider, for example, a direction Ax with AxH(x)Ax < 0 at stationary pow 
x, Quadratic approximation (13.22) implies that 


2 
fx) + AAx) © fx) + Ax H(x)Ax 


< f(x) 


We may conclude that Ax is an improving direction for minimize problems at 
because moves in direction Ax strictly reduce the objective value if A is small enous 
for this quadratic Taylor approximation to dominate higher-order terms. With + 
descent direction at hand, stationary point x could not bea local minimum (princip = 
3.16)). 

The ponderously named positive and negative (semi)definite properties 
square matrices, which are reviewed briefly in Primer 8, address just such sign = 
sues in quadratic forms. Combining with Taylor expression (13.22), we may us 
these properties to distinguish among stationary points. Semidefinite forms provics 
second-order necessary optimality conditions: 


13.3. Derivatives, Taylor Series, and Conditions for Local Optima TAT 


. Similarly, sym- 
are nonzero and 
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We may illustrate principles | 13.21 | and | 13.22/ by testing the two stationan 
points of the example in Figure 13.13 and function (13.18). Using first partial derive- 
tive expressions (13.19), the Hessian at x computes as 


12(a1)? — 24x, 0 = ( 24 0 ) 


mee: ( 0 6 0 6 


The matrix is positive definite because 


ne ( pee )ax =24( Am) +6(Am)y>0 for allAx 40 


Confirming principal | 13.21], local minimum x“ has a positive definite, and thus 
positive semidefinite Hessian. Conversely, we can establish that stationary point x 
is a local minimum by applying principle | 13.22 | with the Hessian positive definite 


The second stationary point x® = (0,5) shows that properties | 


.22 | are not always conclusive. There 


ye 
H(x®) = H0,5) = ( ee a ( ae ) 


and quadratic form 


) ax = 64%)" >0 


We can apply principle |to rule out the possibility of a local maximum becaus: 


this Hessian is not negative semidefinite. Still, principle cannot be applic< 
to assure a local minimum with the Hessian only positive semidefinite. Withow 
extending to third derivatives, we cannot distinguish between a local minimum anc 
a saddle point. 


Verify that function 
fxr, X2, x3) 2 (1)? + x1x2 + 5x2)? + 93 — 2)? 


has a local minimum at x = (0, 0, 2). 
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Analysis: We apply sufficient conditions| 1 ye . First, x must be a stationary point. 


All three partial derivatives 
a a a 
of = 2x, + x, of =x, + 10x, ie = 18(x3 — 2) 
Oxy Chey ax3 
= O atx = (0,0, 2) as required. 
Next we consider the Hessian 


fa leaner () 
H(ix)=]{ 1 10 O 
OPS ORs 


We may verify that this matrix is positive definite, and thus x a local minimum, by 
checking that all determinants of principal submatrices are positive (refer to Primer 
8 if needed): 


2 ail rel: 0) 
det(2)=2>0, det =18>0, det{ 1 10 O | =32430 
ie On, OL, 18 


Verify that function 


fe, %) A Gr)? — 2g — Ge)? 
has a saddle point at x = (1, 0). 


Analysis: To fulfill definition | 13.20], a saddle point must first be a stationary point. 
Checking yields 

a a 

a = 2x, -2=2(1)—2=0 and af = —2x) = —2(0) =0 

Oxy Ox2 
Now computing the Hessian gives 


H(1, 0) = ia = ) 


With first principal determinant = 2, and second = —4, this matrix is neither positive 


semidefinite nor negative semidefinite. Thus x violates requirements] 13.21 | for both 
a local minimum and a local maximum. The remaining possibility is a saddle point. 


13.4 CONVEX/CONCAVE FUNCTIONS AND GLOBAL 
OPTIMALITY 


=e eee Ee ee ed 
Improving search Algorithm 3A (Section 3.2), which provides the paradigm for 
nearly all unconstrained nonlinear programming algorithms, stops if it encounters a 


locally optimal solution (principle |3.6]). What then? We would clearly prefer an 
overall, or global optimum. 

In this section we investigate objective functions having special convex, con- 
cave, and unimodal forms that allow us to prove that a local optimum must also 
be global (see also Section 3.4). With other objectives we must either accept the 
improving search stopping point or try for another by restarting the search from a 
different initial x. 
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Convex and Concave Functions Defined 


Convex and concave functions can be defined in terms of how f(x) changes as = 
move from x“ to x along straight-line path Ax 4 (x® — x), 


Interpolation of f values along the line segment from x") to x@ should neither 
underestimate for a convex function nor overestimate for a concave one. 

Figure 13.15 illustrates for functions of 2-vectors x A (x), x2). The indicate? 
moves start at x and advance toward x®) along direction (x® — x“), Each pois 


(a) Convex 


TIT fT 
A felef 
lel eee 


——— Le 
GD 


(c) Neither 


FIGURE 13.15 Convex and Concave Functions 


13.4 Convex/Concave Functions and Global Optimality FSi 


x in that trajectory has a representation 
x=x 44 _ xD) 


for some A € [0, 1] (property |3.31]). For example, x corresponds to A = 0, and 
x? tox =1. 

The issue in definition| 13.23 |is what happens when we interpolate an estimated 
value for f somewhere along the trajectory. For convex functions [Figure 13.15(a)] 


the corresponding interpolated values 


fea) +2. (fx) — fx) 


should always equal or exceed the true fix” + A(x® —x)). For concave functions 
[part (b)] it should fall equal or below. 

The property must hold for every pair of points x andx®, For example, some 
pairs would meet the test for convexity in Figure 13.15(c), and others would satisfy 
the definition of concave. Still, the function is neither convex nor concave because 
the indicated pair violates both definitions. 


convex, concave, or neither over x € [0, 5]. 


f(s) fe) f(x) 
12 12 12; 
11 11 11 
10 10 10 
9 9 
8 8 8 
u OREM YD: BeBe ThA 15 4 OP 2: Oe Ais z ORS a2 = eS ee iee 5 
x x x 
| (a) (b) ©) 


_ (a) This function is neither convex nor concave. To demonstrate that it is not convex, 
take x =1,x@ =2, anda = 5. 
AE +20 —x) = fd + 42-1) = f.5) © 117 


£ fe) +a (fx) At fx) = 10+ 411-10) = 10.5 


Similarly choosing x“ = 3, x® = 2 and = : establishes that the function is not 
concave because 


fee AGO —x)) 634 102 Sy 5) 10.2 
% fc?) +4 (fe) = fx) = 10+ 41-10) = 10.5 
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(b) This function is apparently concave because definition | 13.23 | holds for all pairs 
of points displayed. 


(c) This function is apparently convex because definition | 13.23 | holds for all pairs 
of points displayed. Notice that convex (and concave) functions need not be differ 
entiable. 


Sufficient Conditions for Unconstrained Global Optima 


The importance of convex and concave objective functions lies with their unusua 
tractability for improving search. 


That is, a search must only achieve a local optimum to produce with no additions 
effort a global minimum of a convex objective function or a global maximum of = 
concave one. 

To see why principle | 13.24 | must be true, consider a convex objective functio= 
f(x), a global minimum x*, and any x that is not globally optimal: 


f(x) 


nonoptimal «© 


x*¥ =x) _ 


Then 


Fee + 2G" —x®) < fe) +2 (fe) - fe) < fle) (13.24 


for all A € (0,1]. That is, direction Ax = x* — x is an improving direction = 
every x) that is not globally optimal. A local optimum, which permits no improvies 
directions, can exist only if it is also a global optimum. 


II ESE 
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Convex/Concave Functions and Stationary Points 


Property | 13.24] shows that we need only compute a local minimum of a convex 
function or a local maximum of a concave one to obtain an unconstrained global 
optimum. In fact, the requirement is even weaker when the objective function is 
differentiable. 


We require only an x with V(x) = 0. 

For an idea of why principle 13.25 | holds, let f be a smooth convex function, 
and Vf(x*) = 0. Convexity definition | 13.23 | assures that 

AK" + A(x — x") < fix") + AFR) — fix’) 
for any x and any 4 € (0, 1]. Furthermore, first-order Taylor approximation 
gives 
FR" + AK — x*)) © F(R") FAV AR) (x - x*) 
Subtracting, simplifying, and dividing by 4 > 0 yields 
F(x) — f(x") = Vix") (x — x*) 

It follows that x* is a global minimum when Vf(x*) = 0 because f(x) — f(x*) = 0 for 
all x. 


SAMPLE EXERCISE 13.13: VERIFYING GLOBAL OPTIMA WITH CONVEXITY _ 
f@) 520 —27 + 6x 


is concave. Use this fact to establish that it has an unconstrained global maximum at 
if = 


_ Analysis: Differentiating yields 
f'@®) =-2x +6 
so that f’(3) = 0 Being a stationary point of a concave function, x = 3 must be an 
- unconstrained global maximum (principle ) 


Tests for Convex and Concave Functions 
Many familiar functions are either convex or concave, but it is often tedious to verify 
definitions} 13.23]. Fortunately, when the function’s domain is all real n-vectors or all 


positive n-vectors, or any other open convex set (definition ), some important 
properties are available to simplify the analysis: 
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1 the domain. 


.|, examine the curve-fitting object»= 


in linear regression form (13.4): 
min f(1,x2) 4 Yo [qi — 1 + mpdP 
i=l 
[The nonlinear case with (q; — x1(p;)* is not convex.] Recall that the p; and q; ar= 


given constants. 
To show this linear regression f is convex, notice first that it is the (unweighted 


sum of functions 
gi(x1,%2) 2 [qi — 1 + opdl 


Under rule} 13.29}, f will be convex if each of the g; is convex. 
Now, dropping the 7 subscripts, we examine 


g(x1,%2) A [g— (1 +xop)P 
= [lq—(@i +moppP 
= [max{(q — x1 — x2p), —(q — x1 — x2p)}]° 
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(the last equality holds because |z| = max{z, —z}). Expressions (g — x; — x2p) and 
—(q — x1 — X2p) are both linear and thus convex by property | 13.28. Therefore, 


A(x1, x2) S |q — 1 + xop| = max{g — x1 — xp, —(q — x, — x2p)} 


is also convex; it is the maximum of convex functions (rule 13 .30 |). Finally, consider 
s(y) Ay’. Second derivative s”(y) = 2 proves s(y) is convex because s”(y) is the 1 by 1 
Hessian matrix and positive definite (property| 13.27] ). Over domain y > 0,s(y) 4 y? 


is also nondecreasing. Thus we may apply composition rule | 13.31 | to conclude that 


g(%1,%2) 2 (q — (ty +x2p))? = s(h(x1, x2) 


is convex. This completes the argument for convexity f. 


ee properiics EE to 13.32 | a to establish that the first two of the following 
functions are convex and the last two are concave over the specified domains. 


(a) fxr, x2) 2 1 + 14 + x12 + (x2 + 1)4 over all x1, x > 0 
(b) fxr, x2) 2 e-3"* over all x1, x2 
(c) flx1, x2, x3) 8 — 4001)* + Sxpx2 — 2(x2)? + 18x3 over all x1, x2 


(d) far, x2) A Sr e 44+ over all x1,.x2 > 0 


Analysis: 
(a) Here the Hessian matrix is 


12(x; + 1)* 1 
Bima) = ( 1 12(x. +1) 


Determinants of its principal submatrices are 12(x +1)? and 144(41 +1)2(a) +1)? — 
which are both positive for all x1, x2 > 0. Thus the Hessian is positive definite, and f 


is convex by rule | 13.27). 


(b) Function A(x1, x2) £ — 3x; +.x2 is convex because it is linear (property 13.28 |). 
Also, g(y) 4 e” in nondecreasing and convex because g”(y) = e” > 0. Thus compo- 


sition rule | 13. 31 


proves that f(x1, x2) = g(h(x1, x2)) is convex. 


(c) For this function the Hessian matrix is 


So 510 
HX, x2, x3) — 5) —4 0 
O20 10 


Principal submatrix determinants are —8, (32 — 25) = 7, and 0, which imply that the 
Hessian is negative semidefinite and f is concave (rule | 13.27 |). 


(d) Over x1, x2 > 0, first term g1(%1,x2) 4 1/(—7x) is the reciprocal of negative- 
- valued, linear, and thus convex function h(x1, x.) 4 —7x1. It follows that this £1(%1, X2) 
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is concave (property | 13.32|). Part (b) already established that go(x1, x2) A e "= 
is convex, meaning (rule | 13.26} ) that its negative is concave. Thus f is the sum of 


concave functions and so concave (property| 13.29 |). 


Unimodal versus Convex/Concave Objectives 


In Section 3.4 we introduced the notion of unimodal objective functions (definition 
[3.26]). A function f(x) is unimodal if f(x) superior to f(x") (greater for a maxi- 
mize, lesser for a minimize) implies that direct move Ax = x — x“ is an improving 
direction at x. That is, heading straight toward any better point should produce 
immediate gain. It follows (principle ) that every unconstrained local optimum 
of a unimodal objective function is a global optimum because improving directions 
exist at every point that can be bettered. 

Since both unimodal and convex/concave objective functions imply that un- 
constrained local optima are global (principles and | 13.24 
should no surprise that there is a connection. 


, respectively), it 


13.33 | Both convex objective functions in minimize problems and concave ob- 
jective functions in maximize problems are unimodal. © 
Expressions (13.23) and (13.24) have already shown why. Improving directions exist 
at all solutions not globally optimal in a convex minimization or concave maximiza- 
tion. 
Unimodality is a weaker requirement than convexity or concavity. 


_13.34 | A unimodal objective function need not be either convex or concave. 


For example, the following is unimodal for a maximize problem, but we showed in 
Sample Exercise 13.12(a) that it is not concave. 


f() 
12 


11 
10 
9 


8 


70 1 2 z 4 5 


Other such examples are our Custom Computer objective in Figure 13.3 and PERT 
example in Figure 13.5. 
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Unfortunately, convenient combination rules such as [13.26 |-[13.32 | do not 
generally hold for arbitrary unimodal objectives. Thus in practice we must often es- 
tablish the more restrictive convex or concave properties to be sure that an objective 
is unimodal. When the functions are not concave for a maximize or convex for a 
minimize, which often happens in applied models, we usually must accept the risk of 
local optima that are not global. 


13,5 GRADIENT SEARCH 


In Section 3.3, principle established that a nonzero gradient Vf(x) provides 
an improving direction at solution x. First-order Taylor series computations of 
expression (13.21) guarantee improvement with sufficiently small steps in direction 
Ax = V/f(x) for a maximize problem or Ax = —Vf(x) for a minimize. In this 
section we develop the simple gradient search algorithm that adopts such gradient- 
based move directions. 


Gradient Search Algorithm 


Algorithm 13D provides details. The move direction for each iteration is derived 
from the gradient at the current point. 


: ® with gradient Vf(x®) #0, gradient search pur- 
suesmove direction 

— : Ax A + Vfx®) 
nimize). 


ALGORITHM 13D: GRADIENT SEARCH 


Step 0: Initialization. Choose any starting solution x, pick stopping toler- 
ance € > 0, and set solution index t — 0. 

Step 1: Gradient. Compute objective function gradient V f(x) at current 
point x, 

Step 2: Stationary Point. If gradient norm || Vf(x®)|| < €, stop. Point x® 
is sufficiently close to a stationary point. 

Step 3: Direction. Choose gradient move direction 


Ax) +V fix®) 
(+ for maximize and — for minimize). 


Step 4: Line Search. Solve (at least approximately) corresponding one- 
dimensional line search 


max or min f(x + ,Ax@D) 


to compute A741. 


Step 5: New Point. Update 
xD ¢ xO 4 Axe) 


Step 6: Advance. Increment ¢ — 7+ 1, and return to Step 1. 
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= (5) 


provides a stopping rule at Step 1 of Algorithm 13D. If the gradient at x is very 
small in length (less than stopping tolerance e), all its components must be very nearly 


zero. Thus the search has essentially reached a stationary point (definition | 13.18 | ). 
Of course, we know from Section 13.3 that a stationary point may be a saddle 


point (definition | 13.20|), not the local optimum we seek. Still, a pure gradient 


algorithm can guarantee no more. When Vf(x) = 0, construction | 13.35 | provides 
no move direction to pursue. 


Gradient Search of Custom Computer Example 


Table 13.5 details application of Algorithm 13D to the Custom Computer regression 
model (13.7) (Section 13.1). Figure 13.16 plots the first few steps on a contour map. 


TABLE 13.5 


~ 


OeOAIDMEWNHF SC 


x 


(32.00, -0.4000) 
(32.00, -0.4687) 
(34.44, 0.4540) 
(34.44, -0.5078) 
(36.28, -0.4962) 
(36.28, -0.5365) 
(37.63, -0.5278) 
(37.63, -0.5571) 
(38.64, -0.5512) 
(38.64, -0.5722) 
(39.36, -0.5684) 
(39.36, -0.5829) 
(39.85, -0.5806) 
(39.85, -0.5902) 
(40.17, -0.5888) 
(40.17, -0.5949) 
(40.38, -0.5941) 
(40.38, -0.5980) 
(40.50, -0.5975) 
(40.50, -0.5997) 
(40.57, 0.5994) 
(40.57, 0.6007) 
(40.62, -0.6005) 
(40.62, -0.6014) 
(40.65, -0.6013) 
(40.65, -0.6018) 
(40.66, -0.6017) 
(40.66, -0.6020) 


The search begins at x© 


[13.35]) is 


f(x) 


174.746 
141.138 
112,599 
93.297 
78.123 
67.897 
60.133 
54.932 
51.006 
48.428 
46.548 
45.348 
44.522 
44.007 
43.672 
43.466 
43.329 
43.248 
43.203 
43.175 
43.160 
43.150 
43.143 
43.139 
43.137 
43.135 
43.134 
43.134 


Vf (x) 


(6.24, 1053.37) 
(-23.06, -0.14) 
(4.57, 759.60) 
(-16.28, -0.10) 
(-3.34, 530.01) 
(-11.34, -0.07) 
(-2.33, 365.74) 
(-7.78, -0.05) 
(-1.48, 251.17) 
(-5.19, -0.03) 
(0.88, 168.22) 
(-3.34, -0.02) 
(-0.51, 108.66) 
(-2.10, -0.01) 
(0.30, 68.07) 

(-1.29, 0.00) 
(0.14, 42. 34) 
(0.76, 0.00) 
(0.10, 24.60) 
(-0.46, 0.00) 
(0.08, 14.77) 
(-0.29, 0.00) 
(-0.04, 9.37) 
(0.18, 0.00) 
(-0.02, 5.78) 
(0.11, 0.00) 
(0.02, 3.37) 
(-0.06, 0.00) 


IVF (x)II 


1053.39 
23.06 
759.61 
16.28 
530.02 
11.34 
365.75 
7.78 
Posey 
5.19 
168.22 
3.34 
108.66 
2.10 
68.07 
1.29 
42.34 
0.76 
24.60) 
0.46 
14.77 
0.29 
9.37 
0.18 
5.78 
0.11 
shot! 
0.06 


Gradient Search of Custom Computer Example 


Neat 


0.00007 
10558 
.00007 
11303 
.00008 
11970 
.00008 
12905 
00008 
3926 
.00009 
14737 
00009 
15220 
00009 
16435 
00009 
15652 
00009 
4805 
00009 


eesoescesessssessessessesesesses 


= (32, —0.4). There the move direction (construction 


Ax = —Vf(x®) = —(—6.24, 1053.37) = (6.24, —1053.37) 


13.5 Gradient Search 


Yd yey ty 
/ Od fp popd ts 


/ 


N 


FicurE 13.16 Gradient Search of Custom Computer Example 


This move direction defines the first line search for step size 4. We identify 
the largest step for which Ax continues to improve by solving the 1-dimensional 
problem 


min f(x + AAx) A f(32 + 6.242, —0.4 — 1053.37A) 


Plotting shows that a minimum occurs at approximately A, = 0.00007. 
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The result is new point 
x) — x 4 V1 Ax © (32, —0.4687) 


Computations in Table 13.5 employ stopping tolerance ¢ = 0.1. Thus the algo- 
rithm continues until norm ||Vf(x®)|| < 0.1 at t = 27. The resulting (approximate) 
stationary point is x°” = (40.66, —0.6020) which we know from earlier analysis 
approximates a local (and here also global) minimum. 
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EXECUTING GRADIENT SEARCH 


SAMPLE EXERCISE 


Consider the unconstrained nonlinear program 


*1 


fae ae 5)? 


max f(x, 
(a) Compute the move direction that would be pursued by gradient search Algorith= 
13D at x — (30,2). 
(b) State the line search problem implied by your direction of part (a). 
Analysis: 
(a) At the specified x, the gradient is 


1 eo = 0.1x,6e%™ 
—0.088 
Vf(30, 2) = (1 4 einy2 ie ( ) 
a 0) 


Thus with a maximize problem we use direction 
Ax = + Vf(x) = (—0.088, 6) 


(b) The line search problem implied by the direction of part (a) is 


(30 — 0.0882) 
max (30 — 0.088%, 2 + 6) 8 7 Sraa-anseny ~ [2 + 64) - 5} 


over A > 0. 


Steepest Ascent/Descent Property 
Gradient search is sometimes called the method of steepest ascent (steepest descent 


for minimize problems) because the direction of construction | 13.35 | produces the 
most rapid rate of objective improvement near the current solution. 


tion Ax A Vf(x) produces the locally 
t, and Ax > <A — Vfix) yields the locally 


steepest rate of objective function as 
steepestrate of descent... 


The search of Figure 13.16 illustrates graphically. The rate of objective im- 
provement at any point depends on the angle between the move direction and near 


objective function contours. Gradient-based directions, which move perpendiculas 
to the contours (principle ), produce the most rapid local progress. 


ve EXERCISE 13.16: COMPUTING DIRECTIONS OF STEEPEST 
ASCENT/DESCENT 


Retna to the nonlinear program of Sample Exercise 13.15 and compute the steepess 
descent direction of length 1 at point x© = (30, 2). 
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Analysis: In accord with principle | 13.36], the steepest descent direction at (30, 2) 
will be the negative of the improving gradient direction computed in Sample Exercise 
13.15(a). Thus the steepest descent direction is Ax = (0.088, —6). 


Zigzagging and Poor Convergence of Gradient Search 


Gradient search is appealingly straightforward, but it is not very effective in most ap- 
plications. To see why, look again at the search in Figure 13.16 and Table 13.5. The first 
move was in direction Ax = (6.24, —1053.37), the second pursued Ax = (23.06, 0.14), 
and the third adopted Ax = (4.57, —759.60). This third direction is almost exactly 
parallel to the first, and the fourth will parallel the second. Later iterations continued 
this zigzagging alternation of almost perpendicular move directions. 

We would prefer to approach the optimum more directly, but these gradient- 
based directions still produced good progress in early iterations. The objective fell 
from 174.746 to 46.548 in 10 steps. 

The difficulty arises later in the search. Near an optimal solution, the shape 
of the objective function changes rapidly with very small step sizes. Thus although 
gradient-based directions produce the locally steepest rate of improvement, they can 
be followed only a very short distance before the best direction changes dramatically. 
The resulting zigzagging consumed the last 17 iterations in Table 13.5 to reduce the 
objective from 46.548 to 43.134. 

Unfortunately, this poor convergence is typical of gradient methods. 


though gradient search may produce good initial progress, zigzagging 
iches a stationary point makes the method too slow and unreliable 
ide Satisfactory results in many unconstrained nonlinear applications. 


Zigzagging is not the only convergence problem with gradient search. With 
small solution changes having a big objective function impact, numerical errors also 
can hopelessly bog down the procedure far from an optimal solution. More sophis- 
tication is required to obtain a really satisfactory improving search algorithm. 


13.6 NEWIONSMETHOD)——s—s—s——s 
Gradient search can be viewed as pursuing the move direction suggested by the 
first-order Taylor series approximation (definition | 13.17 ]}) 

fi (x + AAx) A f(x) +0 fx) - Ax 


Aligning Ax with gradient Vf(x) produces the most rapid improvement in this 
first-order approximation to f(x). 
To improve on the slow, zigzagging progress characteristic of gradient search 


(principle | £3.37 |) requires more information. An obvious possibility is extending 
to the second-order Taylor approximation 


2 
fix® + Ax) 2 FO) +AV AK?) -Ax + TAKA(K Ax 


This section explores the famous Newton’s method, which does exactly that. 
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Newton Step 


Unlike the first-order Taylor approximation, which is linear in directional com- 
ponents Ax;, the quadratic, second-order version may have a local maximum or 
minimum. To determine the move AAx taking us to such a local optimum of the 
second-order approximation, we may fix A = 1 and differentiate f with respect to 
components of Ax. With 4 = 1 the scalar-notation form of fp is 


n af igh ef 
@) A 0) eae ytie, Ih f , 
fox’ + Ax) 3 fx ela eit) Oe (ea) Ax; AX; 


i=1 j=1 


Then partial derivatives with respect to move components are 


af, _ (af) (PF | 
biol. Anorqas wiyoton 
9 Ax; (=) a Baiigy) oy Biagt . 


j=l 


or in matrix format, 


Viz(Ax) = Vix) + Hx )Ax 


Either way, setting Vf:(Ax) = 0 to find a stationary point produces the famous 
Newton step. 


Figure 13.17 illustrates for our Custom Computer curve-fitting model (13.7 


(Section 13.1). First and second partial derivatives at initial point x = (2,-0.4 


/ 
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Figure 13.17 Newton’s Method on the Custom Computer Example 


13.6 Newton’s Method 


are 


~6.240 7agss298708 
On Or 
VAR) = ( 1053 ) cr ee ( 293.99 18,817 ) 


Solving system 
( 7.13 293.99 )ax wack ( —6.240 ) 
293.99 18,817 1053 
produces Newton step Ax = (8.956, —0.1959), which takes us to second-order Taylor 
approximation minimum 
x) =x 4 Ax = (32, —0.4) + (8.956, —0.1959) = (40.96, —0.5959) (13.25) 


unconstrained NLP 


min f(x<1, x2) A (41 + D4 4+ x1%2 + G2 +14 


Analysis: To develop linear system | 13.38], we compute partial derivatives 
4(x, + 13 +22 5 
Vf, 1) = = 
a cower (2) 


12 + 1)” iE 1 
H0. =( seer ye a ) 


Then Newton step Ax is the solution to the system 


rleaes 


which is approximately Ax; = —0.3617, Ax2 = —0.6591. 


and 


Newton’s Method 


Newton’s method proceeds by repeating the process above. That is, it uses first and 
second partial derivatives at the current point to compute a Newton step, updates 
the solution with that step, and repeats the process. Algorithm 13E provides details. 


Newton’s Method on the Custom Computer Example 


Table 13.6 and Figure 13.17 apply Algorithm 13E to our Custom Computer example 
using stopping tolerance « = 0.1. Equation (13.25) already derived the first move 
tox” = (40.96, —0.5959). There derivatives are recomputed, and new Newton step 
Ax = (—0.2733, —0.0061) brings us to 


x? =x + Ax = (40.96, —0.5959) + (—0.2733, —0.0061) = (40.68, —0.6020) 


Notice that a step size of A = 1 is assumed because the Newton step represents a full 
move rather than just a direction. 
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Step 0: Initialization. Choose any starting solution x, pick stopping toler- 
ance € > 0, and set solution index t — 0. 

Step 1: Derivatives. Compute objective function gradient V f(x) and Hes- 
sian matrix H(x”) at current point x®. 

Step 2: Stationary Point. If ||Vf(x)|| <<, stop. Point x® is sufficiently 
close to a stationary point. 

Step 3: Newton Move. Solve the linear system 


H(x®)Ax = — Vf(x®) 


for Newton move Ax“t!), 
Step 4: New Point. Update 


KAD XO 4 Aye) 


Step 5: Advance. Increment t <7 +1, and return to Step 1. 


TABLE 13.6 Newton’s method on the Custom Computer Example 


ae eee 

t x f(x) Vi (x) H(x) IVE (x )II Ax@+) 

0 (32.00,-0.4000) 174.746  (-6.240, 1053) ( 7.13 293.99” 1053.4 (8.956, -0.1958 
293.99 18,817 

1 (40.96, -0.5959) 43.820 (2.347,116.3)  / 4.86 {seat} 116.36 — (-0.2733, -0.006 
166.21 11,564 

2 (40.68, -0.6020) 43.133 (0.0289, 2.989) 4.81 158.97 2.9895 (0.0058, -0.0004 
158.97 10,918 

3 (40.69, -0.6024) 43.133 (0.0000, 0.0027) ( 4.81 158.73 0.00272 Stop 
158.73 10,899 


—_-e_e—————————————————— 


Algorithm 13E stops after one additional move tox® = (40.68, —0.6024). The 
gradient norm 0.00272 at that point is less than stopping tolerance € = 0.1, which 
implies that we are sufficiently close to a stationary point of the full objective function 


Return to the model of eat eerie 13.17, and execute 2 iterations Newton s 
method Algorithm 13E starting with x© = (0, 1). 


Analysis: Sample Exercise 13.17 already computed partial derivative expressions 


40, +13 +x 
V ; = 
f(x1, x2) ( va ie Yd a 
and 
12(x, + 1)? 1 
H = 
Gas X2) ( 1 12 (x2 ze 1)2 
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along with first Newton step Ax = (—0.3617, —0.6591). Thus the first iteration 
produces 


x? = (0, 1) + (—0.3617, —0,6591) = (—0.3617, 0.3409) 


Notice that no step size 1 is applied (or equivalently, 1 = 1). 
Substituting this x in gradient and Hessian expressions produces the next 


13.38 linear system 


4.889 il Ce ae 1.381 
1 28.93 NG Oy) EE 9.282 


There the solution is Ax? = (—0.2184, —0.3133), and we complete the second iter- 
ation with 


x” = (0.3617, —0.6591) + (—0.2184, —0.3133) = (—0.5801, 0.0276) 


Rapid Convergence Rate of Newton’s Method 


Comparison of Tables 13.5 and 13.6 shows dramatically improved convergence with 
Newton’s method versus gradient search. The gradient algorithm required 27 moves 
to reach an optimum. Newton’s method took only 3. 

Although the mathematical theory to fully explain this gain is beyond the scope 
of this book, it is typical of comparative experience with the methods. 


13.39 | If Newton’s method converges to a local optimum, it usually does so in 
many fewer steps than first-order procedures such as gradient Search, 


Computational Tradeoffs between Gradient and Newton Search 


Of course, the number of iterations is not the only consideration in comparing the 
efficiency of algorithms. We must also take into account the effort per iteration. 

There Newton’s method has both advantages and disadvantages. On the pos- 
itive side is the absence of line searches (although some extensions add them to 
Algorithm 13E). Once each direction Ax“ is computed, we may update immedi- 
ately, with no need for a relatively costly search for the best step size A. 

The extra burdens of Newton’s method come with its use of the second-order 
Taylor approximation. Each directional computation on an n-vector x requires 
evaluating n expressions for the various first partial derivatives, and (using symmetry) 
another 5n(n + 1) expressions for the Hessian. This is roughly the same amount of 
work as evaluating the original objective n+ 5n(n+ 1) times versus only 7 for gradient 
search. In addition, we must solve an n by 7 system of linear equations to find the 
next Newton move. This too represents a substantial computational burden at every 
iteration. 
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Starting Close with Newton’s Method 


Perhaps the greatest disadvantage of Newton’s method is that it may not converge 
at all. 


13.41 | Newton’s method is assured of converging to local optimum only if it 


There are two main reasons convergence can fail. First is the quadratic Taylor 
approximation itself. Far from an optimal solution, the second-order approxima- 
tion can give such poor information that the computed Newton step does not ever 
improve the objective function. For instance, suppose that the Newton search of 
Sample Exercise 13.18 had begun at x = (—1, 1) instead of (0, 1). Then the linear 


system of construction | 13.38 | would have been 


0) 1 Ax, ¥ if 

1 48 Ae) eR Cecil 
The implied Newton step Ax” = (—1,17) moves the minimizing objective value 
from f(—1, 1) = 15 to f(—2, 18) = 130,286. Hardly an improvement! 


Another potential difficulty arises with the linear system that must be solved 
at each Newton iteration. How do we know that it can be solved efficiently; that 


(Primer 4), what assures that Hessian matrix H(x) in | 13.38] is nonsingular? 


Near a strict local optimum, second-order sufficient conditions | 13.22 | sugges* 
that the Hessian may be positive or negative definite. Either implies nonsingularit 
(Primer 8). But if we are farther away from an optimum, there is no guarantes 
whatever. 


13.7 QUASI-NEWTON METHODS AND BFGS SEARCH 


In Section 13.5 we saw that gradient search requires only first partial derivatives but 
often gives poor numerical performance. Newton’s method of Section 13.6 yields 
much improved convergence but requires second derivatives and solving a system 
of linear equations at each iteration. It is natural to look for a blend of the two tha: 
preserves their advantages while ameliorating their worst defects, That is preciseh 
the idea behind quasi-Newton methods, which provide the most effective know= 
algorithms for many unconstrained nonlinear programs. 


Deflection Matrices 
The Newton step of construction 13.38 | solves 
H(«)Ax = —V f(x) 


for move Ax. Assuming that the Hessian is nonsingular, we may left-multiply by its 
matrix inverse to express the move as 


Ax = —H1 (x) Vf(x) 


13.7 Quasi-Newton Methods and BFGS Search 


Thatis, directions are computed by applying a suitable deflection matrix D, A H=1!(x) 
to the current gradient. 


By stretching a point, we can also think of gradient search as a deflection matrix 
method. For example, the maximize case of Algorithm 13D employs directions 


Ax = Vf(x®) = —(-DV f(x) 


which can be viewed as adopting negative identity deflection matrix D, = —I. The 
corresponding minimize case uses D, = +I. 


Quasi-Newton Approach 


Quasi-Newton methods work with a deflection matrix that approximates the Hessian 
inverse H~'(x) of Newton’s method. Unlike the full Newton’s method, however, 
this D, is built up from prior search results using only first derivatives. 

The key to this approach is identifying properties that a deflection matrix should 
possess if it is to do the job of an inverse Hessian. Principal among these is the idea 
that the Hessian H(x) reflects the rates of change in first derivatives Vf(x). As 
we move from x to x“+)), it follows that 


Vie) — VARO) SBE) — x0) 
or 


H-1(x) (Vix) = Vicx)) a ge gO 


The analogous requirement on D, is known as the quasi-Newton condition. 


vton algorithms : 


ent change behavio 
condition — 


Another characteristic property of Hessian matrices is their symmetry. For 
most common functions both H(x) and H~!(x) are symmetric matrices. If quasi- 
Newton deflection matrices are to have any hope of approximating such inverse 
Hessians, they must also preserve this property. 
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Guaranteeing Directions Improve 

One of the difficulties that we encountered with Newton’s method in Section 13.6 
is that it gives unpredictable results far from an optimal solution. It may not even 
produce an improving step. 

We would like our quasi-Newton algorithms to avoid this difficulty. Recall from 
earliest principles and that direction Ax improves for a maximize prob- 
lem at x® if Vf(x)- Ax > 0 and for a minimize if V(x) - Ax < 0. With directions 
from deflection matrix construction [13.42], these conditions become 


Vix) (—D, VfK®)) = VAK)(—D) VAK®) > 0 
and 
Vix?)(-—D: Vf) = VAK°)(-D) V(x) < 0 


Notice that the maximize case will be satisfied for any gradient if every D, is 
negative definite, so that —D, is positive definite (Primer 8). Similarly, the mini- 
mize case requires D, positive definite. These concerns motivate another specifica- 
tion. 

13.45 Deflection matrices in quasi-Newton algorithms should assure improving 
directions by keeping D, negative definite for maximize See and | positive 
definite for minimizes. . 


BFGS Formula 


It turns out that a = of | matrix update formulas can meet quas- 


Newton requirements | 13. | 13.43 | to 13.45) 45]. Still, one has proved more effective thar 
all the others. Developed through the combined work of C. Broyden, R. Fletcher 
D. Goldfarb, and D. Shanno, it is known as the BFGS formula. 


13.46) the BEGS 


ula updates deflection matrices by - a. 
D “oe im — dd? ee + dg"D, See 
x) ys = ViK). 


where d 4 x“*) _ x andg A 


Although it appears rather imposing, BFGS update 13.46 actually changes 
deflection matrices rather modestly at each iteration. The update has the form 


D, + ¢C1 — [(D,C2) + (D,C2)"] 
where weight 


D 
guig (13.26 


Beil 
oa1+5e 


| _ 
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is applied to combine simple matrices 


(41)? did, --- didn 
2 eee 
ane daT sear dod; (dz) dodn 
WS de |. ; ae 
dnth. Gnd, aba tae 
(13.27) 
gid, gid, -- gidn 
Os gd” Jona 82d, g2dr +++ godn 
ee Oe ee 
J 
nd) 8nd ee 8nadn 


Notice that both C; and C; are rank one with every row a multiple of every other. 


BFGS Search of Custom Computer Example 


Algorithm 13F details a search algorithm based on BFGS update formula [13.46]. 
Table 13.7 and Figure 13.18 then track its application to our Custom Computer curve- 
fitting model (13.7). 

The initial iteration of Algorithm 13F for this minimize problem employs iden- 
tity deflection matrix Do = I, which leaves the first direction 


Ax® = —Do VfK) = -IVAK) = —Vfx) 


Thus our BFGS procedure follows the same first direction as gradient search Algo- 
rithm 13D. 

Like gradient Algorithm 13D, and unlike Newton Algorithm 13E, quasi-Newton 
methods require line search. Table 13.7 shows that the first such search produces step 
A = 0.0001. The result is 


xD) — x 4 Ax, 
= (32.00, —0.4000) + (0.0001) (6.240, —1053) 
= (32.00, —0.4685) 


with gradient Vf(x™) = (—23.02, 2.514). Thus 


dAx@D _ yo _ { 32.0004 a2) | 0.0004 
early Si ~ \ 0.4685 -04 ] ~ \ —0.0685 


—23.02 —6.240 —16.78 
A EDs. — Ch _ = 
go VARY) — VA) ( 2.514 ( 1053.4 ( —1050.9 


d- g = (0.0004, —0.0685) - (—16.78, —1050.9) = 71.9 


and 
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SI-NEWTON SEAR 

Step 0: Initialization. Choose any starting solution x, compute gradient 
V f(x) and pick stopping tolerance € > 0. Also initialize deflection matrix 

Do = FI 

(— for a maximize, + for a minimize) and set solution index ¢ < 0. 

Step 1: Stationary Point. If norm || Vfx)|| < €, stop. Pointx® is sufficiently 
close to a stationary point. 

Step 2: Direction. Use the current deflection matrix D, to compute the move 
direction 


Ax) © _D,Vf(x®) 
Step 3: Line Search. Solve (at least approximately) 1-dimensional line search 
max or min f(x® +AAx“?) 

to compute step size A;41. 

Step 4: New Point. Update 
XD yO 4 Aged 

and compute new gradient V f(x). 

Step 5: Deflection Matrix. Revise the deflection matrix as 
gD.g\ dd’  D,gd" + dgD, 

d-g/d-g d-g 

where d 4 (x“*) — x) and g A (Vf(x?) — Vf(x)). 
Step 6: Advance. Increment ¢ < 1+ 1, and return to Step 1. 


Dai — D;+ (1 + 


TABLE 13.7 BFGS Search of Custom Computer Example 


t x0 F(x) VF) IVP )II D, ICES) 


1.0000 0.0000 
0.0000 1.0000 
1.0002 -0.0160 
0.0160 0.0003 
0.3759 -0.0059 
-0.0059 0.0002 
0.3775 —0.0055 
3  (4.067,-0.6021) 43.133 (-0.041, 0.100) 0.11 DEE -cIOnne 


(40.69, -0.6024) 43.132 — (0.000, -0.008) 0.01 Stop 


0 (32.00,-0.4000) 174.746 (6.240, 1053) 1053.4 ( (6.240, -1053) 


1 (32.00, -0.4685) 141.139 (23.02, 2.514) 23.15 ¢ (23.06, -0.3684) 
2 (40.66,-0.6068) 43.258 += (-0.823, -51.54) 51.54 f (0.0079, 0.0032) 


(0.0160, 0.0002) 


aN 


Next we compute main update matrices 


dd™ 1 (/ +0.0004 0.0000000 —0.0000004 
d-g —_71.9\ —0.4685 —0.0000004  0.0000652 


Dogd' 1 (1 «20\( -16.78 —0.000095 0.015975 
avg =alo 1 )( oss )(0-0004, 0.4685) (Tp ppsoo7 1.00062 


) 0.0004, —0.4685) = ( 
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FIGURE 13.18 BFGS Search of Custom Computer Example 
Then with 


Teh 30 StO78) \.. 
gDog’ = (—16.78, —1050.9) ( 0 1 ) ( 1050.9 ) = 1,104,583 
the new deflection matrix is 
T T T 
Dy Dy £ 14 SPB) ee tte Po 
d-g/d-g d-g 
( 1 0 ) (1 a) ( 0.0000000 —0.0000004 ) 


0-1 71.9 —0.0000004 — 0.0000652 


Ft: —0.000095 0.015975 —0.000095 —0.005927 
—().005927 1.00062 0.015975 1.00062 


a 1.0002 —0.0160 
~ \ —0.0160 0.0003 


This revised deflection matrix produces the next move direction, 
1.0002 —0.0160 —23.02 23.06 
ql) ay (GD) res ae 
. - PE wieNs ( 0.0160 0.0003 ) ( 2.514 ) i ( 0.3684 ) 
and the search continues. 
Algorithm 13F stops when ||Vf(x)|| < ¢, indicating that we have reached an 


approximately stationary point. Using « = 0.1, this occurs at iteration t — 4 of 
Table 13.7. 


x° = (10,16), Vi) = 4122 oD; = ( Ue ) 


JE Chapter Thirteen e Unconstrained Nonlinear Programming 


of a maximizing search, and then takes a step of Ag = 5 in the BFGS direction toe 
reach a new x with Vf(x®) = (5, —3). 


(a) Determine the direction Ax that was employed and the new solution x, 
(b) Compute the revised deflection matrix De needed for the next iteration. 
Analysis: 

(a) Following gradient deflection computation 


Ax® <— —DsVf(x) = =f a a ) ( . . ( 8 


Thus the new solution 


10 —12 4 
(6). <6) Om 1 = 
x’? = x’ + 16 Ax =( 16) +4( if beige 


(b) We apply BFGS formula | 13.46 |. First, the difference vectors are 


4 10 —6 

d 4x — x6) =" pa) ae) = q 
5 —1 6 

g 4 Vf) — Vf) = ( 3 ) x ( 1 ) a ( ~4 ) 


with d- g = —48. The update matrices are then 


dd™’ 1 /-6 Cb ay 2 (OE: ates 
d-g  —48\ 3 1S No sOiS 59 OMS 75 
Dsgd? 2 ( -10 2 6\63-( 85 425 
d-g — —48 Bo—4 —4 *“000N0.03:5 \ E175 
Also, 
-10 2 6 
gDsg = 6,4) ( : wale. y = -520 


Now substituting in formula | 13.46] gives 
ee dd’ Dsgd' + dg'D; 
d-g /d-g d-g 


Tot 410362: of jee —0.75 0.375 
i 2 —4 —48 0.375 —0.1875 


Pop ee Be sadn ee 
35 bey 4 5s TS 


s ( 4.8750 1.4125 ) 


De Ds +(1+ 


~ \ ~1.3125 —2.7188 


13.7 Quasi-Newton Methods and BFGS Search 


Verifying Quasi-Newton Requirements 
Although proving most of them is beyond the scope of this book, BFGS update 
13.46 | can be shown to fulfill all our quasi-Newton requirements. 


46 | Produces deflection matrices eT 


_BFGS. update formula L 


i-Newton col 


7 13.44 land ae recti speciation Bee 


To illustrate, focus on t = 1 in Table 13.7. There 


Wye an 40.66 \ | 32.00 \ _ 8.66 
—0.6068 —0.4685 —0.1383 
—0.823 —23.02 22.193 
22 2)) _ y= = a2 
ta ee ( 51.54 ) ( 2.514 ) s ( 54.05 ) 
Thus quasi-Newton condition | 13.43 D2g = d checks 
ated 0.37594 —0.00585 22.193 8.66 \ | d 
B= \ _0.00585 0.00016 —54.05 —0.1383 ] 
It is easy to see that all deflection matrices D, in Table 13.7 are also symmetric 


and positive definite (as required for a minimize problem). For instance, with D3 the 
principal minor determinants are 


0.3775 —0.0055 


0.0055 0.0002 = RS eae 


det(0.3775) = 0.3775 >0 and det ( 


Bedasn to the maximize aoa of Saniple Bacicne 13. 19, and domianciias that the 
computed Dg satisfies quasi-Newton algorithm réquirements | 13.43 | to| 13.45 |. 


Analysis: Quasi-Newton condition | 13.43 | is Deg = d. Checking gives 


Bee — SEGio0r = 13125 Ot eee d 
eee Tish eG) Tt 3) 
: : : . e (6) 
as required. Also, matrix De is symmetric (requirement 13.44 ]) because d,, = 


ds} = -1.3125. 
To guarantee an improving direction for a maximize problem, Dg should also 


be negative definite (requirement | 13.45 |). This too is true because principal minor 
determinants 


—1.8750. —1.3125 
det(—1.8750) = —1.8750 and act ( 13125 —2.7188 ) = 3D 


alternate in sign and the first is negative. 


FAS 
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Approximating the Hessian Inverse with BFGS 


Quasi-Newton properties | 13.43) to | 13.45] were motivated to mimic Newton's 
method’s use of Hessian inverse deflection matrices D, = H~'(x®). It should not 
surprise that BFGS and many other quasi-Newton deflection matrices tend to this 
Newton case. 


deflection matrices D, 


Once again, we may illustrate with results for our Custom Computer example. 
The final deflection matrix of BFGS Table 13.7 is 


apse 9803775. , 0.0055 
7 Ewe 200055 0,0002 


The corresponding Hessian matrix is 


Hee ( 4811 158.8 ) 


158.8 10,903 


with inverse 


—0.0058 0.0002 
In accord with principle | 13.48 |, this D3 closely approximates H~! (x). 


Ha) = ( 0.4004 ee 


13.8 OPTIMIZATION WITHOUT DERIVATIVES 
AND NELDER-MEAD 


Sometimes nonlinear programs must be addressed over objective functions that are 
not differentiable, or at least do not have readily computable derivatives. In such 
cases, improving search must rely entirely on functional evaluations. 

How do algorithms choose search directions without derivatives? Numerous 
schemes have been proposed. Some simply use the coordinate directions—searching 
each in turn. Other seek to align the search with the trend of recent progress. We 
develop here only the method due to Nelder and Mead, which constructs directions 


by maintaining an ensemble of current points. 


Nelder-Mead Strategy 


One of the most popular schemes for unconstrained search without derivatives is the 
Nelder—Mead procedure detailed in Algorithm 13G. Table 13.8 traces its application 
to our PERT maximum likelihood model (13.10). 

In contrast to other improving search methods, which keep only one current 
point, Nelder-Mead Algorithm 13G maintains a set of n + 1. 


..,y"", with y having 
the best objective function value, y® the second best, and so on. 


13.8 Optimization without Derivativesand Nelder-Mead 


Step 0: Initialization. Choose (n + 1) distinct solutions x” as starting set 
fy, ...,y"*)}, evaluate fly), ..., f(y@*), and initialize iteration index 
t<—0. 

Step 1: Centroid. Renumber as necessary to arrange the y in nonimproving 
sequence by solution value. Then compute best-n centroid 


1G; 
x = Sy 
nq i=1 


Step 2: Stopping. If all solution values f(y™), ..., f(y”) are sufficiently close 
to centroid objective value f(x), stop and report the best of y and x®, 
Step 3: Direction. Use centroid x to compute away-from-worst move 

direction 


Ax) © xO _ yr 


Step 4: Reflection. Try ) = 1 by computing fix + 1Ax"*), If this new 
value is at least as good as current best f(y"), go to Step 5 and expand. 
If it is no better than second-worst value f(y”), go to Step 6 and contract. 
Otherwise, accept 4 < 1, and proceed to Step 8. 

Step 5: Expansion. Try 4 = 2 by computing f(x +2Ax"*), If this value 
is no worse than f(x® + 1A x") fix 7 <— 2, and otherwise set A <1. Then 
proceed to Step 8. 

Step 6: Contraction. If reflection value fix” +1Ax"*) is better than worst 
current f(y“*?), try 4 = 5 by computing f(x +4Ax"*)), If not, try A = —} 
by evaluating fix — 3Ax‘*?), Either way, if the result improves on worst 
current f(y"*)), fix A at the +5 tried and proceed to Step 8. Otherwise, go 
to Step 7 to shrink. 

Step 7: Shrinking. Shrink the current solution set toward best y by 


y? — f(y +y®) for alli =2,...,2+1 


Then compute new fiy),..., fly@*?), advance t — t+ 1, and return to 
Step 1. 
Step 8: Replacement. Replace worst y"*" in the solution set by 


x9 4 AXED 


Then advance t <t+1 and return fo Step 1. 


Each iteration of the search tries to replace the worst solution y"*” with a better 
one. 
The maximization of Table 13.8 illustrates for n = 2. Search begins with the 
ensemble 
yD), y” = 66, 3), y® = ©, 4) 
fy) = 12.425, fly) = 11.429, fry®) = 2.663 


Notice that the solutions are numbered from best to worst. 
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TABLE 13.8 Nelder—Mead Search of PERT Example 


t yD y? y?) y? Ax) Reflect Second 
0 (5.000, 3.000) (6.000, 3.000) (6.000, 4.000) (5.500, 3.000) (-0.500,-1.000) %=10 [K=05 
f=12425 f=11429 f=2663 f=13.084 fall4ls f=24.915 
1 (5.250, 2.500) (5.000, 3.000) (6.000, 3.000) - (5.125, 2.750) (-0.875, -0.250) |A=1.0 N= 2.0, 
f=24915  f'=12405 f=11429 f=13.074 f= 30.005 f= 19.654 
2 (4.250, 2.500) (5.250, 2.500) (5.000, 3.000) (4.750, 2.500) (-0.250,-0.500) 2=10 [A=05 
f=30005 f=24915 f=12425 f=30.482 f= 18.229 f=17.354 
3 (4.250, 2.500) (5.250, 2.500) (4.625, 2.250) (4.750,2.500) (0.125,0.250) 2=10 2=0.5 
f=30.005  f=24915  f=17:354 f= 30.482 f=21.434 f= 11.231 
4 (4.438, 2.375) (4.750, 2.500) (4.250, 2.500) (4.594, 2.438) (0.344,-0.063) [K=1.0 
f=35513 f=30482 f=30.005  f=16.119 f= 31.642 
5 (4.438, 2.375) (4.938, 2.375) (4.750, 2.500) (4.688, 2.375) (-0.063,-0.125) 2%=10 A=-05 
f=35513  f=31642 f=30482 f= 15.893 f=17354 f=26.577 
6 (4.438, 2.375) (4.594, 2.438) (4.688, 2.375) (4.516, 2.406) (-0.172,0.031) [k=1.0 
f=35:513 f=16.119 f= 15.893 f= 25.980 f=31.062 
7 (4.438, 2.375) (4.344, 2.438) (4.594, 2.438) (4.391, 2.406) (-0.203,-0.031) A=1.0 A=0.5 
f=35513  f=31062 f=16119 f=32.845 f=17253 f=28811 
8 (4.438, 2.375) (4.344, 2.438) (4.289,2.391) (4.391,2.406) (0.102,0.016) A=10 [A=0. 
f=35513  f=31.062 ~ f=28811 7 232.845 f=29.758 f=31.979 
9 (4.438, 2.375) (4.441, 2.414) (4.344, 2.438) (4.439, 2.395) (0.096,-0.043) =1.0 [A=—05 
f=35513  f=31.979 ,, f=31.062 . f=33,857 f= 26.094 f= 32.250 
10 (4.438, 2.375) (4.392, 2.416) (4.441,2.414) (4.415, 2,396) (-0.027,-0.019) [A= 1.0 
f=35:513 f = 32.250 f=31.979 f= 33.726 f=34.140 
11 (4.438, 2.375) (4.388, 2.377) (4.392, 2.416) (4.413, 2.376) (0.021, -0.040) A=1.0 A= 2.0 
f=35513 f=34140 f=32250 f=34.957 f=37.893 f= 41.050 
12 (4.455, 2.296) (4.438, 2.375) (4.388, 2.377) (4.446, 2.335) (0.058, -0.042) =A) A= 2.0 
f=41.050 f=35513 f=34140 f=38.569 f=42813 f=27.929 
13. (4.504, 2.294) (4.455, 2.296) (4.438, 2.375) (4.479, 2.295) (0.042, -0.080) A= 1.0 
f=42813  f=41.050 f=35.513 f=43.202 f= 41.193 
14 (4.504, 2.294) (4.521, 2.215) (4.455, 2.296) (4.513, 2.254) (0.058,-0.042) A=10 ([R=-05 
f=42813  f=41193 f=41.050 f=42.388 f=28.723 f= 44.727 
15 (4.484, 2.275) (4.504, 2.294) (4.521, 2.215) (4.494, 2.285) (-0.027,0.070) 2=1.0 [a=-0.5 
foiddga7 fa 42819. ef=41.193 Fa 45.087 f=37.793 f= 44.627 
16 (4.484, 2.275) (4.508, 2.250) (4.504, 2.294) (4.496, 2.262) (-0.009,-0.032) [A=10] 2=2.0 
f=4a707-- f=4aen7 f= 42813 f= 46.671 f=46.356 f= 43.832 
7 (4,487, 2.231) (4.484, 2.275) (4.508, 2.250) (4.485, 2.253) (-0.022,0.003) 2=10 [a=—05 
f= 46.356 f=44.727 f= 44.627 f= 45.790 f=42708. f= 47.301 
18 (4.497, 2.251) (4.487, 2.231) (4.484, 2.275) (4.492,2.241) (0.008, -0.034) A=1.0 X= 2.0 
f=47301 f=46356 f=44727 f=46.977 f= 48.520 f= 44,839 
19 (4.500,2.207) (4.497, 2.251) (4.487, 2.231) (4.498,2.229) (0.011,-0.002) a=10 [a=-05 
f=48520 . f=47301 . 246.356. f=48.192 f=44882 f= 47.275 
20 (4.500, 2.207) (4.497, 2.251) (4.493, 2.230) (4.498, 2.229) Stop 


f=48.520 f=47301 f=47275 f=48.192 


Sy See. eee ek en ee ee ee Se eee eh ee ee 


13.8 Optimization without Derivativesand Nelder-Mead 


It is somewhat arbitrary that exactly n + 1 solutions are maintained in the 
ensemble. Still, too many would complicate computation, and too few would not 
adequately surround an emerging optimum. Over n decision variables, n +1 solutions 
is just enough to define vertices of a polytope surrounding a point. 


SAMPLE EXERCISE 13.21: ARRANGING A NELDE -MEAD ENS! 


Consider the unconstrained nonlinear program 
min f(x1, x2, %3) 4 Qa)? + G2 — D)* + 3 +4)? 
Choose an initial ensemble of solutions for Nelder-Mead search and arrange them 


with the notation of | 13.49 |. 


Analysis: With n = 3 we need 4 distinct solutions that “bracket” a region likely to 
produce good solutions. One possibility is 


oh CMe ated 2) 


Evaluating the objective gives 
73,3, 3) = 62, FE3,3, =3)= 14, ¥GN0}3)= 59, and f(—3,0,-3)=11 


Thus, for this minimizing model, we have initial ensemble 


3 =3 3 3 
y? ee 0 |, y? wld Sth y =| 0], and y” 241 63 
—3 =3 3 3 


Nelder-Mead Direction 


The critical issue in any derivative-free optimization method is how to construct 
search directions without the aid of partial derivatives. Nelder-Mead Algorithm 
13G adopts an away-from-worst approach. 


Nelder-Mead 1 vith employs 


= 50 : At ite ation ¢, The 


The idea is to move away from the worst solution in the ensemble and toward the 
rest. 

Figure 13.19 illustrates for t = 0 of the 2-variable search in Table 13.8. The 
centroid of the best 2 solutions is 


PGneles 


AVY 
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- 


FIGURE 13.19 Nelder-Mead Direction for PERT Example 
Thus the search direction of | 13.50 | is 
SS) 6 —0.5 
A Oey a ch 
oe eae 


SAMPLE EXERCISE 13.22: COMPUTING NELDER-MEAD DIRECTIONS 
Return to Sample Exercise 13.21, and compute the Nelder-Mead direction corre- 
sponding to the initial ensemble constructed there. 


Analysis: The centroid of the best 3 solutions in that 3-variable model is 
xO =1 (yD 4 y@ 4 yO) 
= $[(-3, 0, -3) + (-3, 3, -3) + G, 0, 3)] 
a (ails 1, —1) 


Thus direction | 13.50 | becomes 


Ax =x —y® = (-1,1, -1) — G, 3, 3) = (-4, -2, -4) 


Nelder-Mead Limited Step Sizes 
Centroid x plays the role of a current solution in Nelder-Mead Algorithm 13G. Be 
what step size A should be applied to direction | 13.50 |? 
One strategy would undertake a full line search over 
x + Ax 
However, the required computation is usually not justified for such a crudely derive: 
direction Ax. Also, too large a step would destroy the “spread” pattern of 


ensemble. 
Algorithm 13G confronts these concerns by trying only one or two i’s dra 


from possibilities {+1, +2, +5, —}}. 


aR Se ae | 
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13.51| The Nelde 


~ further tria 


Figure 13.19 illustrates for t = 0 in Table 13.8. Trying 4 = 1 produces reflection 

point 

x + Ax = (5.5, 3) + (—0.5, -1.0) = (5, 2) 
with f(5,2) = 11.415. The term reflection comes from the fact that this new point 
mirrors worst ensemble y® = (6, 4) on the opposite side of centroid x = (5.5, 3). 

Immediately replacing y® with this reflection point would leave it worst in the 
new ensemble. Thus we try to do better by contracting with 1 = +4, (Contraction 
would use A = —} if the reflection point had been no better than worst y®.) 

The resulting (5.25, 2.5) yields far superior (5.25, 2.5) = 24.915. Substituting 
this contraction point for the worst in the current ensemble produces the improved 
set 

y= (§.25,2.5), y® = (5,3), y® = (6, 3) 
fly) = 24.915, fly) = 12.425, fly) = 11.429 


oft = 1. Notice that points have been renumbered to keep them in objective function 
value sequence. 

Table 13.8 shows the direction for this new ensemble is Ax = (—0.875, 
—0.250), and new centroid x" = (5.125, 2.75). Now reflection with A = 1 produces 
new best (4.25, 2.5) with (4.25, 2.5) = 30.005. This leads to trying expansion with 
4 = 2. However, the result is not as good, so the reflection point joins the ensemble. 


which (if any) chosen. 


SK - 4xA) f+ 4xA) fK41Ax) fK +2 Ax) 


(a) iW 15 18 30 
(b) 40 22 12 10 
(c) 43 51 60 25 
(d) 60 49 44 70 


Analysis: We follow details of Algorithm 13G. 


a) Reflection (A = 1) objective value 18 is neither better than best ensemble value 
‘iy) = 13 nor worse than second worst f(y) = 25. Thus we replace worst current 
point y with the one for A = 1. 
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(b) Reflection (A = 1) objective value 12 is better than ensemble best f(y) = 13. 

Thus we expand and try 4 = 2. The resulting value of 10 improves, so we replace 

worst current point y™ with the one for 4 = 2. | 
| 
| 


(c) Reflection (A = 1) objective value 60 is worse than ensemble worst f(y“) = 50 
Thus we contract and try A = —}. The resulting value of 43 is better than worst, so 
1 


we replace y with the one for 4 = —}. 


(d) Reflection point (2 = 1) objective value 44 is better than worst fry) = 50, but 
not second worst f(y) = 25. Thus we contract and try A = 5. The resulting value 
of 49 is better than worst, so we replace y® with the one for A = }. 


Nelder-Mead Shrinking 


Sometimes neither the reflection point nor a contraction alternative yields a solutio= 
that would much improve the ensemble. This suggests the need for rescaling the 
entire array of ensemble points. 


Figure 13.20 depicts such a shrinking step at t = 3 of the PERT maximus 
likelihood search in Table 13.8. Ensemble points y? and y® are moved halfway 
best y as 


wer -worevrai[(8)(8)] -() 


new y® <— 3(y® +y®) = 5 ( “= ) au ( ne )| FS ( 2395 


After renumbering to reflect new objective value sequence, the search proceeds wa® 
this more compact ensemble. 


Suppose that it becomes necessary to apply shrinking Step 7 of Algorithm 13G wos 
current ensemble 


yO =(2,3), y=(-4,5), and y® =(8,-1) 


Compute the new ensemble. 


Analysis: We apply computatio 


new y® = y® = (2, 3) 
new y® = 3(y® + y®) = 4[(2, 3) + (-4, 5)] = (-1, 4) 
new y9 = 3(y + y®) = 4 (2,3) + (8, -D] = 6, 1) 


Fe NI et) 
Le We Ee oo? KY’ =new yy", 
77 yy) = new y?) 
- 4 
yg 2 
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Exercises 


FIGURE 13.20 Nelder—Mead Shrinking in PERT Example 


Nelder-Mead Search of PERT Example 

Table 13.8 detailed all 20 iterations required in Algorithm 13G search of our maxi- 
mum likelihood PERT model (13.10). When one of the trial 2 values was adopted, it 
was shown boxed in the table. Otherwise, shrinking procedure | 13.52 | was invoked. 


Nelder-Mead search stops when objective function values for points in the 
ensemble become essentially equal. In Table 13.8 this condition was reached when 


n+1 


n+1 


DY [fy®) — ff <€=0.5 
i=l 


Then the best of the ensemble and centroid points is selected as an approximate 


optimum x* = (4.5, 2.207) with f(x*) = 48,520. 


EXERCISES 


13-1 A biomedical intrumentation company sells its 
main product at the rate of 5 units per day. The in- 
strument is manufactured in lots run every few days. 
It costs the company $2000 to setup for production 
of a lot and $40 per unit per day to hold finished in- 
struments in inventory between runs. The company 
would like to choose a lot size that minimizes aver- 
age inventory and setup cost per day assuming that 
demand occurs smoothly at the given rate. 


(a) [Q] Formulate a 1-variable unconstrained NLP 
to choose an optimum lot size. 

(b) [| BIPlot the objective function of your model 
and compute an optimum lot size graphically. 


13-2 As part of a study of 911 emergency calls, an 
analyst wishes to choose the value of parameter « in 
exponential probability density function d(t) 4 ae~* 


that best fits call interarrival times 80, 10, 14, 26, 40, 
and 22 minutes. 


(a) Formulate a 1-variable unconstrained NLP to 
choose a maximum likelihood estimate of a. 

(b) [Q] i Plot the objective function of your model 
and compute an optimal estimate graphically. 


13-3 An oil drilling company wishes to locate a sup- 
ply base somewhere in the jungle area where it is 
presently exploring for oil. The base will service 
drilling sites at map coordinates (0,-30), (50,-10), 
(70,20), and (30,50) with helicopter supply runs. The 
company wishes to choose a location that minimizes 
the sum of flying distances to the four sites. 


(a) [9] Formulate an unconstrained NLP to choose 
an optimum base location. 
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(b) [S| & Use class optimization software to com- 
pute at least a local optimum starting from co- 
ordinates (10,10). 


13-4 [6] Repeat Exercise 13-3, this time minimizing 
the maximum distance to any drilling site. 


13-5 An electronics assembly firm is planning its 
production staff needs to make a new modem. It has 
measured one test worker assembling the unit and 
observed the following data: 


Through unit 2 6  =20 25255540 
Averagetime 84 55 42 3.7 3.1 


Experience shows that learning curves, which 
describe the ability of a worker to improve his or her 
productivity as more an more units are produced, 
often take the form 


average time = a (no. units)? 
where a and b are empirical constants. 


(a) [] Formulate an unconstrained NLP to fit this 
form to the given data so as to minimize the sum 
of squared errors. 

(b) [S| & Use class optimization software to com- 
pute at least a local optimum for your curve fit- 
ting model starting with a = 15, b = —0.5. 


13-6 The following shows a series of measurements 
of the height (in inches) of a new genetically engi- 
neered tomato plant versus the number of weeks af- 
ter the plant was replanted outdoors. 


Week 1 2 4 6 Sao 
Height 69! s 1554922, 33). 44s 52 


Researchers wish to fit this experience with an 
S-shaped logistics curve 
k 


SS re eatb(weeks) 


where k, a, and b are empirical parameters. 
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(a) Formulate an unconstrained NLP to fit this form 
to the given data so as to minimize the sum oF 
squared errors. 

(b) | & Use class optimization software to com- 
pute at least a local optimum for your curve 
fitting model starting with k = 50, a = > 
b = —0.3. 


13-7 The university motor pool’ provides a large 
number of cars n for faculty and staff traveling or 
university business. Motor pool cars have an aver 
age annual cost of f dollars per car for fixed expenses 
such as depreciation, insurance, and licensing, plus « 
variable operating cost of v,, cents per mile driver 
Those travelers who cannot- be accommodated 5 
the pool must drive their personal cars and be reim- 
bursed at v, cents per mile. Demand varies great) 
among times of the year, but an extensive analysis 

past travel records has fitted regression equations 


mn) & am + bmn + en/n 
p(n) La, +byn+ep/n 


to the annual numbers of miles that would be drives 
in motor pool and personal cars as a function of th 
size of the motor pool available. Formulate a 
variable unconstrained NLP to compute a minimus 
total cost motor pool size. Ignore integrality. 


13-8 Once a site for a new service facility has bees 
chosen, the limits of its market area must be detes 
mined,’ along with the corresponding facility six: 
Assume (i) that the facility is to be located at 1% 
center of a circular market area and sized to co 
uniform density d calls per unit area out to a radius 

r; (ii) the cost of operating the facility is [f + c(size 
where f is a fixed cost and c a variable cost per us 
size; and (iii) transportation costs per call are prop 
tional to (straight-line, one-way) distance from ™: 
facility to the customer at ¢ per unit distance. Fs 
mulate a 1-variable unconstrained NLP to choose « 
market area radius that minimizes the facility’s aver 
age total cost per call. Evaluate any calculus integrs 
in your objective. 

13-9 Renewing highway pavement markings> oo 
c dollars per mile but reduces social costs fos 
delays, accidents, and other effects of declines 


3Based on W. W. Williams and O. S. Fowler (1980), “Minimum Cost Fleet Sizing for a University 


Motor Pool,” Interfaces 10:3, 21-27. 


4Based on D. Erlenkotter (1989), “The General Optimal Market Area Model,” Annals of Operations 


Research, 18, 45-70. 


>Based on V. Kouskoulas (1988), “An Optimization Model for Pavement Marking Systems,” Euro- 


pean Journal of Operational Research, 33, 298-303. 


marking performance over time. Suppose that new 
markings yield maximum performance pmax and that 
performance ¢ days after renewal can be expressed 
P(t) & Pmaxe™, where a is a constant depending on 
the durability of the markings. Also assume that 
each unit of lost performance costs d dollars per day 
per mile. Formulate a 1-variable unconstrained NLP 
to choose a time between pavement renewals to min- 
imize average total daily cost per mile. Evaluate any 
calculus integrals in your objective. 


13-10 The number of potential patrons p; of a new 
movie theater complex has been estimated from 
census data for each of the surrounding counties 
i = 1,...,15. However, the fraction of potential 
patrons from any 7 who will actually use thé com- 
plex varies inversely with its (straight line) distance 
from the county centroid at coordinates (x;, y;). For- 
mulate a 2-variable NLP to choose a maximum total 
realized patronage location for the complex. 


13-11 Denoting by n, the number of universities us- 
ing a textbook through semester ¢ of its availability 
(no = 0), the number of new adoptions in any sin- 
gle semester ft can be estimated (a+bn,_1)(m—n,_1), 
where a and b are parameters relating to the rapidity 
of success, and m is the maximum number of univer- 
sities who will ever adopt. 


(a) Given values of n, for t = 1,..., 10, formulate 
an unconstrained NLP to make a nonlinear least 
squares fit of the foregoing approximation to 
these data. 

(b) [] Show that an appropriate change of param- 
eters can covert your model into a linear least 
squares problem. 


13-12 Major aircraft parts undergo inspection and 
overhaul? every f; flying hours, and replacement ev- 
ery ty. Experience shows the cost of overhauling a 
particular model of jet engine can be expressed as 
increasing nonlinear function a(t,)? of the overhaul 
cycle, and operating costs per hour are an increasing 
nonlinear function c(t)“ of the replacement time. 
Each replacement costs a fixed part cost f plus labor 
cost g(t,)" to identify and fix associated problems. 
Formulate an unconstrained NLP over t, and & to 
find an overhaul and replacement policy that mini- 
mizes total cost per flying hour. 


13-13 Determine whether each of the following func- 
tions is smooth on the specified domain. 
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Exercises 


(a) Bl fe) 4x4 + 3x —19 forall x 

(b) f(x) A max{2x — 1,2 — x} for all x 

(c) &] f@) Alx—5| forx > 0 

(d) f(x) £3x+In() for x > 0 

(ec) BI fei, x2) 2 xe” for all x1, x5 

(£) flr, x2) & xy — 1] + [x2 +3] for x1, x2 > 0. 


13-14 Each of the following plots shows a function 
f(x). Determine graphically whether each indicated 
point is an unconstrained local maximum, an un- 
constrained global maximum, an unconstrained lo- 
cal minimum, an unconstrained global minimum, or 
none of the above over the domain depicted. 


(2) & 


f() 
A 


| | 
| | 
| I 
| | 
| | 
| | 


xO) 


x4) x) x(6) 


(b) fe) 


13-15 Each of the following plots shows contours of 
a smooth function f(x), x2). Determine graphically 
whether each indicated point is an unconstrained lo- 
cal maximum, an unconstrained global maximum, 
an unconstrained local minimum, an unconstrained 
global minimum, or none of the above over the do- 
main depicted. 


(a) BQ] Points x = (1,3), x = (4,4), x® = 
(3, 3),x® = (4, D,x® = (2,1) 


Based on T. C. E. Cheng (1992), “Optimal Replacement of Ageing Equipment Using Geometric 
Programming,” [International Journal of Production Research, 30, 2151-2158. 
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--20 
a 
y x4) 


I 
\ 0. 
N\ 


(b) Points x9 = (6,6), x® = (2,8), x® = (8,2), 
x® = (4,4),x® = (7, 6) 


13-16 Determine graphically whether each of the 
following functions is unimodal for a maximize 
problem and/or for a minimize problem over the 
range depicted. If not, identify a violating pair of 
points. 


(a) The function of Exercise 13-14(a) 
(b) The function of Exercise 13-14(b) 

(c) [| The function of Exercise 13-15(a) 
(d) The function of Exercise 13-15(b) 

(ce) EK] The function 


fx) 


(f) The function 
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Fx) 


13-17 [O] Use golden section Algorithm 13A to find 
an optimum of the NLP 


70 
min 10x + — 
x 


ste: Wai 10 


to within an error of 41. 


13-18 Use golden section Algorithm 13A to find a= 
optimum of the NLP 


max 500—x(x —20)3 
s.t. 0<x<20 


to within an error of +1. 


13-19 Suppose that we were given only the lowes 
limit of 1 in the NLP of Exercise 13-17. Apply 3-poi 
pattern Algorithm 13B to compute a corresponc- 
ing upper limit with which golden section searc® 
could begin using each of the following initial ste> 
sizes 6. 


(a) BI 6=0.5 
(b) &] 6=16 


13-20 Do Exercise 13-19 for the NLP of Exercs: 
13-18 using 5 = 2 and 6 = 20. 

13-21 [O] Use quadratic fit Algorithm 13C to com 
pute an optimum for the NLP of Exercise 13-~ 
within an error tolerance of 2. Start with the 3-po= 
pattern {1,2,10}. 

13-22 Use quadratic fit Algorithm 13C to compe: 
an optimum for the NLP of Exercise 13-18 with= 
an error tolerance of 4. Start with 3-point pattie 
{0,3,20}. 

13-23 Consider the 1-variable function f(x) 2 © — 
3x? + 11x at current point x = 3. 


(a) [Q] Derive the first-order Taylor approximati 
to f(x +A). 

(b) [| Derive the second-order Taylor approxim= 
tion to f(x +A). 

(c) & Plot the original function and both Taylor = 
ries approximations in the vicinity of x. How 
accurate do the approximations appear to ™ 
Which is better? 


13-24 Do Exercise 13-23 for function f(x) A 18x — 
50In(x) at x = 16. 


13-25 Consider the 2-variable function 
f(%1, X2) & 041)? — Sxix2 + 6(x2)? with current point 
x = (0,2) and move direction Ax = (1, —1). 


(a) [| Derive the first-order Taylor approximation 
to f(x + AAx). 

(b) [Q] Derive the second-order Taylor approxima- 
tion to f(x + AAx). 

(c) i Plot the original function and both Taylor 
series approximations as functions of A. How 
accurate do the approximations appear to be? 
Which is better? 


13-26 Do Exercise 13-25 for function 
FO1,%2) 5 13x, — 6x1x. + 8/m,, x = (2,1) and 
Ax = (3,1). 


13-27 For each of the following unconstrained NLPs, 
either verify that the given x is a stationary point of 
the objective function or give a direction Ax that im- 
proves at x. 


(a) &] min (x1)? + xx. — 6x, — 8x2, x = (8, —10) 
(b) max 10(x1)? + 12 In(), x = (1, 2) 

(c) ] min 16x, — x1x2 + 2(x9)?, x = (3, 0) 

(d) max x,x2 — 10x, + 4x2, x = (—4, 10) 


13-28 For each of the following functions f, use con- 


ditions| 13.19 | to] 13.22 | to classify the specified x as 


definitely local maximum, possibly local maximum, 
definitely local minimum, possibly local minimum, 
and/or definitely neither. 


(a) LQ) fer, x2) 2 301)? — x12 + (2)? — 11, x = 
(2, 1) 

(b) fi, x2) & — (1)? — 6x1 x2 — 9(%2)”, x = (-3, 1) 

(c) El fi, x2) Ax) —xyx2 + (2)? x = (2,1) | 

(d) f(x1, x2) & 12x. — (41)? + 3x12 - 302)?, x = 
(12, 8) 

(e) EI fr, x2) 4 x10), x = (0, 0) 

(f) f(xy, x2) 2 6x) + In@y) + @)*, x = (1, 3) 

(g) Ke fx ,X2) A 2ci)? t 8x1X2 + 8(x2)* 12x, 
24x2,x = (1, 1) 

(h) ft, x2) A 401)? + 3/x2 — 8x) + 3x2, x = (1, 1) 


13-29 Determine whether each of the following func- 
tions is convex, concave, both, or neither over the 
domain specified. 


(a) [9] fs, x2) A In(x)) + 20 In(aa) over x1, x) > 0 
(b) f(x) 4 xsin(x) over x € [0, 27) 
(c) [O] fx) Ax — 2) over all x > 0 
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(d) f(x) A @ —8)* + 132x over all x 

(e) [Ol fea, ...,%5) 4 3x1 + LL — x3 — 8x5 over all 
(QB eg U5) 

(f) f%1, x2) 2 21x, + 63x over x1, x. > 0 

(g) [Ol fr, x2) S max{x;, x2} over all x1, x 

(h) f(xy, x2) A min{13x,—2x2, —(x2)*} over all x1, x9 

(i) EO) fr, x2) 2 10/7 — 40/9 over x1, x) > 0 

(j) fer, x2) 2 In[—301)? — 9@2)?] — 10/x) over 
X1,X2 > 0 


13-30 Use convexity/concavity to establish that each 
of the following solutions x is either an unconstrained 
global maximum or an unconstrained global mini- 
mum of the f indicated, and explain which. 


(a) [Q] foi.) A Qa — 5)? +m + ( — 7)? at 
Xe = (2510) 

(b) fxr, x2) £ 500 — 8(x, +1)? — 20) — 1)? + 4xyxy 
at x = (—1, 0) 


13-31 Consider the unconstrained NLP 
max x1,X) — 5(x, —2)4 —3(%) —5)4 


(a) Use graphing software to produce a contour 
map of the objective function for x; ¢€ [1, 4], 
x2 € [2, 8]. 

(b) ] Compute the move direction that would be 
pursued by gradient search Algorithm 13D at 
gO = Cla). 

(c) [Q] State the line search problem implied by 
your direction. 

(d) [| & Solve your line search problem graphi- 
cally and compute the next search point x. 

(e) [| Do two additional iterations of Algorithm 
13D to compute x® and x®), 

(f) Plot progress of the search on the contour 
map of part (a). 


13-32 Do Exercise 13-31 for the unconstrained NLP 
1000 


+ (x1 — 4)? + (a — 10)? 
My a Xe 


min 


starting from x = (10, 1), and plotting x, € [2, 11], 
x € (0, 15]. 

13-33 Return to the unconstrained optimization of 
Exercise 13-31 starting from x = (3,7). 


(a) [| Write the second-order Taylor approxima- 
tion to the objective function at x for unknown 
Ax andi’ =1. 

(b) [6] Compute the Newton direction Ax at x 
and verify that it is a stationary point of your 
second-order Taylor approximation. 
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(c) &] Beginning with your Newton direction, 
complete 2 iterations of Newton’s method Al- 
gorithm 13E. 

(d) &Plot progress of your search on a contour map 
like the one of Exercise 13-31(a). 


13-34 Do Exercise 13-33 on the NLP of Exercise 
13-32 starting from x = (10, 1). 

13-35 Return to the unconstrained optimization of 
Exercise 13-31 and consider BFGS Algorithm 13F 
starting at x = (2, 3). 


(a) [] Compute the first direction that would be 
pursued by Algorithm 13F. 

(b) [] Assuming that the optimal step is 4 = 0.026 
in that direction, compute the new solution x", 
the next deflection matrix D, and the next BFGS 
search direction Ax. 

(c) Verify that your deflection matrix of part (b) sat- 
isfies quasi-Newton conditions (within 
roundoff error) and is symmetric. 

(d) Verify algebraically that your direction of part 
(b) is improving. 

(e) © Plot your first move and second direction on 
a contour map like that of Exercise 13-31(a). 


13-36 Do Exercise 13-35 on the NLP of Exercise 
13-32 starting from x© = (8, 1) and using A = 0.32 
in part (b). 
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13-37 Consider the unconstrained NLP 
min max{10—x; — Xx, 
6 + 6x, — 3x2, 
6 — 3x, + 6x2} 


(a) [|] Explain why Nelder—Mead scarch is appro- 
priate for solving this unconstrained optimiza- 
tion. 

(b) [8] Do 3 iterations (moves) of Nelder-Meaé 
Algorithm 13G, starting from initial ensemble 
(5,0), (10,5), (5,5). 

(c) Plot the progress of your search through cen- 
troids x and connecting the 3 points of eace 
ensemble with dashed lines. 


13-38 Do Exercise 13-37 for the NLP 


max min{20—x,—%x%, 6+3x,-m, 6—x,-— 


starting with ensemble (0,0), (0,2), (2,2). 

13-39 Compute the Nelder-Mead Algorithm 13¢ 
ensemble that would result from applying the shrink- 
ing step to each of the following (y best objective 
value, etc.). 


a) By® = 42,1, y? = 64,5), y¥O = 
3,2,7), y® = (7, 2,7) 

(b) y® = 10, 8, 10), y® = (4, 6, 2), y = ©, 0. 
y” = (0, 10, 6) 


Luenberger, David G. (1984), Linear and Nonlinex 
Programming, Addison-Wesley, Reading, Mass. 


Ce ictal ies One Ep 


Constrained 
Nonlinear Programming 


In this chapter we introduce models and methods for constrained nonlinear opti- 
mization. We begin by modeling a series of real applications, emphasizing differ- 
ences between linear and nonlinear constraints, convex and nonconvex feasible sets, 
separable and nonseparable objective functions, and so on. Then a variety of solution 
methods are developed, some fairly general purpose and others restricted to special 
classes on NLPs. Theoretical development draws on Chapters 3, 5 and 13. 


14.) CONSTRAINED NONLINEAR PROGRAMMING MODELS 


As usual, we begin our investigation of constrained nonlinear programming with 
some examples. In this section we formulate three cases, illustrating the broad range 
on constrained NLP, and in Section 14.2 we present three more, representing some 
classic special cases. All are based on real application contexts drawn from published 
reports. 


EXAMPLE 14.1: BEER BELGE LOCATION ALLOCATION 


One common class of nonlinear programs arises from the location and customer al- 
location of distribution and service facilities. The task confronted by a large Belgian 
brewery we will call Beer Belge illustrates. Beer Belge wishes to realign its 17 depots 
to more efficiently solve its 24,000 customers throughout Belgium. For purposes of 
this analysis the customers can be aggregated in 650 regions. 

Assigning index dimensions 


EiGepot number (f= 1, ....... 17) 


j & customer region number (j = 1, ..., 650) 


company analysts can determine from maps and past experience 


1Based in part on L. F. Gelders, L. M. Pintelon, and L. N. Van Wassenhove (1987), “A Location- 
Allocation Problem in a Large Belgian Brewery,” European Journal of Operational Research, 28, 196-206. 
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hj A x-coordinate of the center of customer region j 
k; & y-coordinate of the center of customer region j 
d; & number of delivery trips required to region j per year 


We wish to choose locations for the depots and allocate customer region demand to 
minimize total travel cost. 


Beer Belge Location-Allocation Model 


As with all such location-allocation problems, decision variables in a model of Beer 
Belge’s case must settle two distinct but related questions. First, we need to know 
where the depots will be located. Then we must allocate the trips required in various 
regions among the depots. The following address both: 


x; 4 x-coordinate of depot i’s location 
y; 4 y-coordinate of depot i’s location 
w;; & number of trips per year from depot i to customer region j 


To obtain an objective function, we adopt the common assumption that round- 
trip travel cost from depot 7 to customer region j is proportional to the straight-line 
(Euclidean) distance between their locations. Then minimizing total transportation 
cost of the location-allocation amounts to 


min a aye (number of trips from i to j)(distance from i to /) 
noah 
Using symbolic constants and decision variables defined above, we obtain 


17 650 
min Se x wins) Ox = ane + (yi zs kj)? 
t=1) j=1 


To complete a model, we must add constraints assuring that all required trips 
to each region are made. The result is 


17 650 
min Ss ye wisn (xi = hj)? + (i — kj)? (total travel) 
i=l j=l 


17 (14.1 
s.t. yy wij = dj J HAES TR 650 (allocate trips) 

=A 

wij = 0 (i — A loser 7/2 js ce aon!) 


SAMPLE EXERCISE 14.1: FORM 


Two airstrips are to be constructed in the jungle to service 3 remote oil fields. The 
first oil field requires 25 tons of supplies per month. The second, which is 75 miles 
east and 330 miles north of the first, requires 14 tons. The third, which is 225 miles 
west and 40 miles south of the first, needs 34 tons per month. Formulate a location- 
allocation model to locate and operate the airstrips to minimize the ton-miles flows 
per month. 


Modeling: Coordinates in this problem may be introduced by taking the first oil fielc 
as (0, 0). Then the second is (—75, 330) and the third is (225, —40). 


Jesgass oo oe eee net ip Se ee ee et ee ee ee See 


14.1 Constrained Nonlinear Programming Models 


Decisions involve both the location of the airstrips and allocation of workload be- 
tween the strips. Thus we employ decision variables (x1, y;) 4 coordinates of the 
first airstrip, (x2, y2) A coordinates of the second, and wi; & tons flown from airstrip 
i to oil field 7. The resulting model is 


min wy1/ (1 — 0)? + (y; — 0)? + Wi 2 (x1 + 75)? + (¥1 — 330) (ton-miles) 
+ W13y/ (1 — 225)? + (y1 + 40)? + wo.1/G2 — 0)? + Gr — 0)2 
+ W,2¥/ (x2 +75)? + (¥2 — 330)? + wo3/Org — 225)? + Gn +40) 
St. W114 + W214 = 25 (field 1) 
W142 +W22 = 14 (field 2) 


W13 + W23 = 34 (field 3) 
w;; >O0i=1,2; jes 


Here the objective function totals flight distances times tons shipped, and constraints 
allocate needed shipments among airstrips. 


Linearly Constrained Nonlinear Programs 


Location-allocation model (14.1) illustrates the many large-scale nonlinear programs 
having all constraints linear, Only its objective function renders the model an NLP 

Linearly constrained nonlinear programs form an important special class be- 
cause much of the tractability of linear programs extends to cases with only the 
objective function nonlinear. Fortunately, linearly constrained NLPs are also very 
common, 


‘nearly all their constraints linear. : 


EXAMPLE 14.2: TEXACO GASOLINE BLENDING 


Beginning as early as the 1950s, oil companies have used mathematical programming 
to help plan gasoline blending at refineries. Texaco is no exception.” 

Figure 14.1 sketches gasoline processing in refineries. Crude oil is first distilled 
into a range of materials from light ones such as gasoline to much heavier ones such 
as fuel oil. The “straight-run” gasoline produced in distilling is not nearly enough to 
meet market demands economically. Thus most of the distillates that are heavier or 
lighter than gasoline are re-formed and cracked to produce other gasoline forms. A 
final processing stage combines some additives with these various stocks of gasoline 
to produce market blends (e.g., premium unleaded, regular unleaded) with suitable 
quality indexes (e.g., octane, lead content, sulfur content, volatility). 

Typical operations plan blending for a month at a time. Using index dimensions 


i A gasoline or additive stock number ae es 8 
j 4 market blend number (j = 1,...,7) 
k A quality index number (k = 1,..., h) 


2Based on C. W. DeWitt, L. S. Lasdon, A. D. Waren, D. A. Brenner and S. A. Melhem (1989), 
“OMEGA: An Improved Gasoline Blending System for Texaco,” Interfaces, 19:1, 85-101. 
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additives ———_——__» 


straight-run 
gasoline 


gasoline blending 
market blends 


FiGurE 14.1 Gasoline Refinery Flow 


We will assume that the following constants are known as each optimization begins 


p; = estimated selling price per unit of output blend j 
r; 4 quantity of output blend j required 

s; £ quantity of input stock i available 

v; & estimated cost per unit of input stock i 


ai, & kth quality index of input stock i 
£x & lowest acceptable level of quality index & in output blend j 
uj, x & highest acceptable level of quality index & in output blend j 


We want to find a maximum profit (sales income minus stock costs) plan to meet all 
blending requirements with available stocks. 


Texaco Gasoline Blending Model 


Clearly, the decisions in gasoline blending involve how much of which stocks to us 
Thus we will develop a model with decision variables 


x;; 4 quantity of input stock i used in output blend j 
In accord with principle , much of the model will be linear. We begi= 


with the objective to maximize sales income minus stock cost: 


max 2 53 (Dj — Vi)xij (profit) 


i=l j=l 
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Linear constraints enforce stock availabilities and output demands: 
n 
ae (ie ret 771 
j=l 


m 
) xij 27; a ey 
i=1 


Most of the blending constraints also take the linear form of the blending LPs 
in Section 4.2. That is, we require the average level of each quality index to fall 
between upper and lower limits for each output blend with constraints 
Dien GikXij 

ye Xi 

What introduces nonlinearity in gasoline blending are two classes of quality 
indexes that do not combine linearly. The volatility quality measures perform loga- 
rithmically to produce constraints 


TL 
ag ; ie 
ein <In (AsAés) Saige HT Ae” volatility 
dem ij 
Octane measures are even more complex. Typical schemes use known constants b; x, 
Cik, dix, and e;;, to specify octane limits with fourth-order expressions 


ei DiRxiy a were” 7 om Fie ij)? 
(tae m 2 3 
dint Mi (ons %is) (or xi) 
Ee ene)” 
m 4 
(or xi) 
Collecting all the above and adding variable-type constraints completes the full 
gasoline blending model: 


lik < SU pe hak oa. linear fe 


lik XS 


+ < Uj jJ=1,...,n; octane k 


+ > a (Pj — VAXij (profit) 
i=l j=l 
‘S$ Xij <5i i=1l,...,m (availability) 
i 
aye ji lalein (demand) 
i=1 
bin < Sa Sy j=1,...,n; lineark (blend 1) 
dint Hj (14.2) 
epsetn( te | a, 7 =A de Golalility. R (blend 2 
J, > nea ‘D ; 
i=1 4) 


ee int DikXij i ea) ena) 
Die Hy Ores Cras) 
eras) es a 
ha “rd 


egy =O (el Ime (7) fale n2M 


jJ=l,...,m; octane k (blend 3) 
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This time it is the last two systems of main constraints that make the model a nonline= 
program. All other elements are linear. 


Engineering Design Models 


The mostly linear location-allocation [model (14.1)] and gasoline blending [mode 
(14.2)] nonlinear programs treated so far are typical of large-scale applications. Stil 
the NLPs arising in engineering design often have quite a different character. 


EXAMPLE 14.3: OXYGEN SYSTEM ENGINEERING DESIGN 


To illustrate the smaller but more nonlinear models arising in engineering, we com 
sider the design of the oxygen production system for a basic oxygen furnace in th 
steel industry.* Figure 14.2(a) shows the main components of the system. The oxyge 
plant produces at a constant rate, with output compressed and stored in a tank. The 
furnace then uses the oxygen in a cycle like the one depicted in Figure 14.2(b). Re- 
atively low demand level d applies during the first t, minutes of the cycle, followes 
by a burst to demand d from time ¢, through h. 


furnace 
demand 


compressor 


storage 
tank 


furnace 
—> > time 
St ED 
(a) System components (b) Operating cycle 


FIGURE 14.2, Oxygen System Engineering Design Example 


We seek a minimum cost sizing of the system components. In particular, we wish ° 
choose values for 4 decision variables: 


3Based on F.C. Jen, C. C. Pegels, and T. M. DuPuis (1967), “Optimal Capacities of Product 
Facilities,’ Management Science, 14, B573-B580. 
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x, £ production rate of the oxygen plant 
x2 A pressure in the storage tank 

x3 4 compressor power 

x4 4 storage tank volume 


Also, physical limitations require that the storage tank pressure be at least Po. 


Oxygen System Engineering Design Model 


The procurement and operation costs of this oxygen system has 4 parts: 
ae us foe ) is ae) i be: a “a ae ) 
cost plant cost cost cost power cost 
Using prior experience with similar systems, engineers in this application estimate 
oxygen plant cost to be linear in the production rate as 
oxygen plant cost = 61.8 + 5.72 (production rate) 
Compressor costs grow with compressor power in the nonlinear function 
compressor cost = 0.0175 (power)”* 

Storage vessel costs perform similarly in the required volume 

storage tank costs = 0.0094 (volume)””° 


Finally electricity for the compressor is proportional to the power and time of oper- 
ation as 


electrical power cost = 0.006 (time of operation)(compressor power) 


One constraint of the model comes from the requirement that the oxygen pro- 
duction rate outputs at least what is required over the cycle, that is, 


x1 > dit + dr(t — t1) 
Also, the minimum pressure po imposes bound 
x2 = Po 


The remainder of the constraints enforce the physical relations among the de- 
cision variables. First, the power of the compressor is related to the pressure that 
it must generate in the storage tank. In particular, the compressor power must ad- 
dress the maximum stored inventory (d) — x1)(h — t;) just before the burst part of 
the operating cycle. Including standard temperature and gas constants gives the 


constraint 
dy — to —t 
ie 36.25! 2 — X1)(t — ty) In (2) 
ty Po 


Finally, we must relate storage volume to the pressure needed to keep the 
maximum required inventory. Using appropriate temperature and gas constants 
gives 


(dz — x1) (b — fh) 
x2 


X4 = 348,300 
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Collecting all the above and adding variable-type constraints, we obtain the 
following NLP model of our oxygen system design problem: 


min 61.8 +5.72x; + 0.0175(x3)° + 0.0094 (x4)975 


+ 0.006t)x3 (total cost) 
St. x1 > dit, + do(h — th) (demand) 
*2 = Po (pressure) 
d (143 
PSgehe 36.25! 2—x1)(b — hh) in (2) tas =) 
al Po pressure 
dy — =f 
Xi 348,300 62 =~ = 4) es a 
x2 pressure 


X1, X2,X3,X4 > 0 
With constants 
d=2.5, d=40, 4=06, t=1.0, po = 200 


an optimal design uses production rate xj = 17.5, storage pressure x} = 473.7, com 
pressor power x3 = 468.8, and storage volume x; = 6618 for total cost approximate 
173.7. 


SAMPLE EXERCISE 14.2: FORMULATING ENGINEERING DESIGN MODELS 


A closed cylindrical tank is being designed to carry at least 20 cubic feet of chemica!s 
Metal for the top and sides costs $2 per square foot, but the heavier metal of th: 
base costs $8 per square foot. Also, the height of the tank can be no more than twice: 
its diameter to keep it from being top heavy. Formulate a constrained nonlines 
program to find a design of minimum cost. 


Modeling: The decision variables the designer must choose are 
x; 4 diameter of the tank 


X2 A height of the tank 


Then we obtain the NLP 


2 2 
min 2 (aux. + aad +8 (« oY ) (metal cost) 


2 
Ruger eh (volume) 


st. 2 
X2 < 2x1 (height-to-diameter ratio) 
x1 > 0, x2 = 0 


The objective function minimizes metal cost for the sides and top at $2 per squar: 
foot, plus cost for the base at $8. One main constraint makes the tank have th: 
required volume, and the other enforces the height-to-diameter ratio limit. 


Te | a Se ES rm ee 
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14.2, CONVEX, SEPARABLE, QUADRATIC 
AND POSYNOMIAL GEOMETRIC PROGRAMMING 
SPECIAL NLP FORMS 


Linear programming is one especially tractable special case of constrained nonlinear 
programming, but it is not the only one. Several other common forms have special 
characteristics that can be exploited in search algorithms. 

In this section we define and illustrate models in four of these unusually tractable 
classes: convex programs, separable programs, quadratic programs, and posynomial 
geometric programs. Then in Sections 14.7 to 14.9 we illustrate how their special 
properties can be exploited. As usual, the models presented are drawn from real 
application contexts, usually cases described in published accounts. 


EXAMPLE 14.4: PFIZER OPTIMAL LOT SIZING 


One common application context yielding NLPs with special structure arises in man- 
aging inventories and choosing manufacturing lot sizes. Work at Pfizer, a large man- 
ufacturer of pharmaceuticals,* offers a good example. 

The production of pharmaceuticals involves a series of fermentation and or- 
ganic synthesis steps done in large tanks holding several thousand gallons. Each 
“campaign” or lot of a product consists of one or more “batches” processed serially. 

The main issue to be decided is how many batches should be included in a 
campaign or lot of each product. Large changeover times and costs are incurred each 
time a new campaign starts because of the need to clean and reconfigure equipment 
carefully for the next product. These changeover burdens must be balanced against 
the cost of holding inventories built up during campaigns. In particular, rigorous 
quality control requires that all batches of a campaign be held at the end of each 
manufacturing step until every one is ready for the next step. 

Our specific (fictitious) version of this lot sizing problem will consider products 
j=1,...,4, each having steps k = 1, 2 of production (k = 0 refers to raw materials). 
Table 14.1 details values of the associated input parameters 

dj 4 number of batches of product j demanded annually 
t,x & changeover time per campaign of product j at step k (in weeks) 


Pix & process time per batch of product j in step k (in weeks) 


I> 


Vj, = value per batch of product j at the end of step k (in thousands of dollars) 


TABLE 14.1 Pfizer Lot Sizing Example Data 


Changeover Process 


Product Annual Time (weeks) Time (weeks) Value ($000) 
Number, Demand, 


J dj fr G2 Pian Pia = V0 Vit. Vio 
1 150 is Og ie ee ek 
2 220 13 20 AO aS geo. 70 — 10 
3 55 63 62 D5 FAI AR 208 A 
4 90 0.9 18 Mes 35g aA 60 v8 


“Based on P. P. Kleutghen and J. C. McGee (1985), “Development and Implementation of an Inte- 
grated Inventory Management Program at Pfizer Pharmaceuticals,” Interfaces, 15:1, 69-87. 
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We will also assume that 3000 total production weeks are available to manufacture 
the 4 products, that holding inventory cost 5% per week of the product value, and 
that changeover activities cost c = $12,000 thousand per week. 


Pfizer Optimal Lot Sizing Model 


To model our version of the Pfizer application, introduce decision variables 
x; & number of batches of product j in each campaign or lot 
To begin, notice that the implied number of campaigns per year is 


annualdemand — dj 


campaign size Xj 


Then the total annual cost objective function can be expressed as 


4 5 
a 
“ne ‘5 dj ter aie sae) fe aun costs Pra) (144 


x; | \per campaign of j campaign of j 


Changeover costs for product j are easily expressed as 
2 
changeover costs 
2 3 .f=Hc Sy bik 

per campaign of j = 
To compute the corresponding inventory holding costs, consider the cycle i 
lustrated in Figure 14.3. Each campaign of product j begins with x; batches valuec 
at raw material cost vo. Over the xjp;1 weeks it takes all batches to complete the 
first step of processing, their value increases to x;v; 1. Then the next XjPj.2 weeks raise 
the value to xjvj2. Finally, the finished pharmaceutical is distributed to customers 
over the 52/(d;/x;) weeks until the next campaign is completed. Inventory cost per 


campaign is then 5% of the area under the implied inventory value curve. 
Substituting in expression (14.4) gives the complete objective function 


4 2 2 
; d; . 52v; 
min Sa = . > ti + 0.005 2 SDik Vie + vj R) Qj)? + ae wt) 
jan k=1 J 


ie— 


The only main constraint in this model will enforce the limit on production time 
available. Again summing per campaign, we have 


4 s A . : 
d; | (changeover time roduction time time 
bo jo E < 
fai *%j L\per campaign of i per campaign of j/ | — \ available 
or 


4 d 2 2 
Dee ( tha + Sou < 3000 (145 
es k=1 k=1 


jel] 


Substituting parameter values of Table 14.1 and simplifying produces the com- 
plete model 


—oCCM 
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inventory 
value 
A 


all batches 
value after Se mie 
step 2 


all batches 
value after 
step 1 


raw 
material 
value 


time 


step 1 


: step2 _ distribution 
production production time between 
time for time for campaigns 


all batches all batches 


Ficure 14,3 Pfizer Lot Sizing Example Inventory Cycle 


2160 12 
min 66.21x, + = + 426.8x2 + sh (total cost) 
1 2 


330 2916 
61.20 — + 268.1 — 
+ x3 + rs an X4+ %, (14.6) 


180) * 726". 275, +243 
ee 1 9915 (production time) 
x4 x2 x3 xX4 


Mio 0) 


s.t. 


An optimal plan runs 
4, = F161... x = 3003, %, 2911, . 41 = 4135 (14.7) 


campaigns per year of the 4 products. Total annual cost will be approximately 
$6,837,000. 


Convex Programs 
Lot sizing model (14.6) is special, in part, because it forms a convex program. 


97 
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In Section 13.4 we defined convex and concave functions (definition 13.23) 


and showed their importance in characterizing the most tractable objective functions. 
Convex programs extend these ideas to constraints. Every > constraint should be 
concave (after collecting terms involving the decision variables on the left-hand side). 
every < constraint convex, and every = constraint linear. 

To see that lot sizing model (14.6) satisfies these requirements, we first consider 
its objective function: 


8712 


X2 i 

330 2916 i 

Oh 20i4 = — + 268. 1x, 4+ = — 
XB X4 


2160 
min f(x) 4 66.21x; + pil + 426.8x2 + 
x1 


Notice that it is a sum of terms 


aX; + e 
with positive constants a and £;. The first such terms are linear and so convex 
(principle ). But linear functions are also concave, which makes reciprocal 
terms convex for x; > 0 (principle [13.32]). It follows that the objective function is 
a sum of convex functions, and so itself convex (principle ). This is just what 
is needed in definition : 
Now consider constraints 


pepe onan ee 
x4 x2 x3 X4 

g(x) 4x, >0 . 

g(x) Sx > 0 7 

ga(x) 4x3 > 0 


g5(x) 4x4 >0 


Definition requires g; convex and g>,..., gs concave. Function g; conforms for 
much the same reason as the objective: It is the positive-weighted sum of reciprocals 
of nonnegative linear functions. Other functions g», ..., gs are concave because they 
are linear. 


program. 


(a) max 3w,—w2+8In(w4) 
S.t. 4(w 1)? —WywW2+ (w2)? < 100 
wy tw, =4 
W1,W2 > 0 
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(b) min 3w; —w2+8In(wj) 
st. (Wi)? + (Wo)? > 10 
Wi +wWw.=4 
W1,W2 > 0 
(c) max w,+7w2 
s.t. W1W2 < 14 
(w1)? + (w2)? = 40 
W1,W2 >0 
(d) min w,+7wo 
s.t. W1 +wW2 < 14 
W1—-W2 => 0 
2w, +5w2 = 18 
W1,W2 > 0 


Analysis: We apply definition ; 
(a) The objective function of this model is concave because it is the sum of a linear 


function and 8 In(w), which has negative second derivative (principle Asai | ). Also, 
the first main constraint function is convex because the Hessian matrix 


8 -1 
a ( Bay 2 ) 
is positive definite (principle | 13.27]). Since all 3 other constraints are linear, the 


model is a convex program. It maximizes a concave objective, subject to a convex < 
constraints, a linear = constraint, and concave (because linear) > constraints. 


(b) This NLP is not a convex program because its objective function, which is the 
concave one of part (a), isinappropriate for a minimize. Also, the first main constraint 


(w1)* + (wa)? = 10 
is convex in a > form. 


(c) This model is also not a convex program. Its first main constraint involves function 
&1(W1, W2) = W1W2, which is neither convex nor concave. Its Hessian matrix is 


Bim) =({ 0) 


Furthermore, the second main constraint is a nonlinear equality. 


(d) This model is a linear program and so a convex program. Linear objectives 
and constraints, which are both convex and concave (principle | 13.28]), fulfill all 


requirements of definition ’ 


Special Tractability of Convex Programs 


To appreciate the special tractability of convex programs, reach back to our charac- 
terization in Section 3.4 of the models most convenient for improving search. The 


best objective functions are those that are unimodal (definition ), and principle 
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| 13.33 | has already established that maximizing a concave function, or minimizing a 
convex, produces an objective with that desirable property. 
For constraints we would like the implied feasible set to be convex (definition 


). Requirements for a convex program assure exactly that. 


144 The feasible set defined by constraints = - 


= 
a bj i=, 77 


Bi(X) 


is convex if each g; of a = cons 
convex, and each g; of an = 


We can see why property is true by considering a single convex constraint 
g(x) <b 


and two points x and x) that satisfy it. If the corresponding feasible set is to be 
convex, every point along the line segment between x and x™ must also satisfy the 


constraint, that is (principle ), every point representable as 
x) + A«@ — x) 
for some A € [0, 1]. Using 
g(x) <b 
g(x) <b 
we may multiply the first by (1 — 4), the second by A, and sum to conclude that 
(1 —A)g@x) + Ag(K) < 1 —2)b + (Ab 


which simplifies to 
gx") +4 (gx) — gx) <b 


With g convex, the left side of this expression is at most g(x + A(x® — x)) (defi- 


nition | 13.23)). Thus 
g(x + Aa — x) <b 
which shows that the line segment point for 1 is feasible are required. A similar 
argument demonstrates that feasible sets for concave > constraints are convex, and 
we already know that linear constraints yield convex feasible sets (principle | 3.32 
Combining the unimodality of objective functions for convex programs with the 
4.|) to their main convenience 


Improving search procedures that produce locally optimal solutions automatically 
yield global optima. 


ee ee es Sd ee ee ee gee ee a ee 
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Separable Programs 


Besides being a convex program, the objective and constraint functions of Pfizer lot 
sizing model (14.6) also have a special separable property. 


That is, a function is separable if it is the sum of 1-variable functions of its arguments. 
To see that Pfizer objective function (14.8) is separable, consider the 1-variable 
functions 


fii) 4 66.2141 + a 
1 


fala) 4 426.8%) + a= 
2 


fa(x3) & 61.20x, + = 
3 


fala) & 268.1%, + 2° 
X4 
Clearly, the full objective f(x) separates into a sum of these functions of single decision 
variables. 
Similar definitions establish that constraint functions g(x) through gs(x) are 
separable. For example, 


81(%) = 81,1(%1) + 81,2(%2) + 81,3(%3) + B14 (X4) 


where 81,1(X1) = 180/x1, 81,2(x2) = 726/X2, 81,3(X3) = 27.5/x3, and &1,4(%4) = 243 /x4. 
When both the objective function and all constraint functions are separable in 
this way, we term an NLP a separable program. 


Pfizer model (14.6) provides an example. 
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SAMPLE EXERC ARABLE PROGRAMS — / = 


Return to the NLPs of Gamcle eres 14.3, and determine whether each i is a sepa- 
rable program. 


Analysis: We apply definition | 14,7 |. 


(a) The objective function and most constraints of this model are separable. However. 
it fails the definition of a separable program because the first main constraint contains 
the term —w4w2, which involves both variables. 


(b) This model is a separable program. Each objective and constraint function can be 
expressed as a sum of separate functions, one involving only w, and the other only w= 


(c) Again, the objective and most constraint functions of this NLP are separable 
Still, the first main constraint is not, because it includes the term w,w2. Thus the 
model is not a separable program. 


(d) This model is a linear program. By definition, linear functions involve sums o! 
constant multiples of the decision variables. Thus all are separable, and every linear 
program is a separable program. 


Special Tractability of Separable Programs 


Separable programs, where objective and constraint functions are sums of 1-variable 
functions, have much in common with linear programs where the corresponding 
functions are limited to scalar multiples of the decision variables. This relationship 
can be exploited by piecewise-linear approximation of each component function as 
suggested in Figure 14.4. That is, each of the single-variable components of the 
objective and constraint functions is approximated by a series of linear (straight- 


f(y) = y2-12y +45 


0 1 2) 3 4 5 6 U 8 9 


FIGURE 14.4 Piecewise-Linear Approximation of a One-Variable 
Function 
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line) segments. Then, if the separable program also conforms to convex program 
definition | 14.3 |, the approximation can be solved by linear programming. 


straint. Chuneuats 


Details are provided in Section 14.8. 


EXAMPLE 14.5: QUADRATIC PORTFOLIO MANAGEMENT 


We may illustrate another important class of constrained nonlinear programs by re- 
turning to an application setting that we encountered earlier in Chapter 8: finance. 
Financial managers are constantly planning and controlling market decisions, and a 
wide array of nonlinear programs have been employed to help. 

We will consider a simple case of portfolio management—dividing investment 
funds to maximize return and minimize risk. Our manager, whom we will call Barney 
Backroom, must decide how to split the available funds among three classes of in- 
vestments: common stocks, money markets, and corporate bonds. Table 14.2 shows 
the experience of the past 6 years on which Barney will base his decisions. He would 
like to average an 11% return on investments while accepting minimum risk. 


TABLE 14.2 Return Experience for Portfolio Example 


Percent Returns for Year: 


Category 1 2 3 4 5 6 Average 
Stocks 22.24 16.16 5.27 15.46 20.62 -0.42 13.22 
Money market 9.64 7.06 7.68 8.26 8.55 8.26 8.24 
Bonds 10.08 816 846 918 9.26 9.06 9.03 


Quadratic Portfolio Management Model 
The obvious decision variables for a model of this problem are 
x, 4 fraction of the portfolio invested in common stocks 


X) 4 fraction of the portfolio invested in money markets 
x3 4 fraction of the portfolio invested in corporate bonds 


Then one main constraint forces all funds to be invested: 
Meo Ll 


One reasonable assumption about returns is that each class will achieve the 
average experienced in the years of Table 14.2. Then the goal of returning 11% can 
be expressed as 


13.22x1 + 8.24x2 + 9.03x3 > 11 


The more difficult issue is how to model variability of return. One measure 
is the variance—the average squared deviation from the mean. If the three classes 
of investments varied independently, the variance of the overall return would be 


803 
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simply the sum of the variances of each class. However, financial markets in various 
commodities tend to interact. Thus a workable model should include the covariances 
relating movement in the different categories of investment. 

Given a series of n observations like those in Table 14.2, covariances can be 
estimated as 


v,; & estimated covariance between categories i and j 
ty aS i value j value pin hn y i value s j value ) 
Ten = inperiodt/\inperiodt) 2 = in period t = in period t 


Then the variance of the total return is approximately 


n nA 
variance of return = y y Vi jXiX; 
i=1 j=l 
= xVx 


for V is the matrix of v;;. 
Table 14.2’s data for our example yields 


66.51 2.61 2.18 
Vi 2.61 0.63 0.48 
2.18 0.48 0.38 


Thus we may combine with the constraints above to obtain the model 


min 66.51(x1)? + 2(2.61)x1x2 + 2(2.18)x;x3 + 0.63(x2)? (variance) 


+ 2(0.48)x2x3 + 0.38(x3)? 
SiseMGs ep Exs = "1 (invest 100%) (14.1 
13.22x1 + 8.24x> + 9.03x3 > 11 (return) 


X1,X2,%3 > 0 


An optimal portfolio invests a fraction x} = 0.47 of funds in common stocks, x5 = 
of funds in money markets, and x3 = 0.53 in corporate bonds to produce a minimum 
variance of 15.895. 


Quadratic Programs Defined 


Model (14.10) illustrates the special case of a quadratic program. 


Portfolio model (14.10) clearly qualifies. All 5 constraints are linear, and th: 
objective function involves only squares and products of 2 variables. In the mat 


format of definition , its objective function has 
c= 0, Q=V 


14.2 Convex, Separable, Quadratic and Posynomial Geometric Programming Special NLP Forms 805 


that are, also place the objective function in matrix format e- w + wQw. 


(a) max 3w, — 5w2 + 12(w1)? + 8wiw2 + (Ww)? 
s.t. Wi t+w2=9 
W1,W2 = 0 


(b) min wyw2w3 
st. (wi)? + (W2)? < 25 
(c) min 5w;+19w2 


Se Wwytw,=9 
W1,W2 = 0 


Analysis: We apply definition 3 


(a) This model is a quadratic program because its objective function involves only 
second-order terms, and its constraints are linear. Here 


Srl) anand) 


Notice that the coefficient 8 on cross-product ww is split so that 


11,2W1W2 + q21WiW2 = 4wiw2 + 4wiw2 = 8wiw2 


(b) This model is not a quadratic program. Its objective involves a 3-way product, 
which is not quadratic, and its constraint is not linear. 


(c) This model is a linear program and so quadratic. Objective function elements 
Hi oie 
oe ( 19 ) syn 


Special Tractability of Quadratic Programs 


Being nonlinear in only the objective function, and only quadratic there, quadratic 
programs have much in common with LPs. In particular, powerful dual and com- 
plementary slackness properties make possible efficient specialized algorithms for 
many cases. Section 14.7 provides details. 


EXAMPLE 14.6: COFFERDAM DESIGN 


To illustrate another special type of constrained NLP arising often in engineering 
design applications, we consider the planning of a cofferdam.> Cofferdams are used 
to block streams temporarily while construction is in progress. Figure 14.5 illustrates 
a common design. Each “cycle” of the dam consists of a large steel cylinder filled 
with soil and joined to the next with curved connecting plates. 


>Based on F. Neghabat and R. M. Stark (1972), “A Cofferdam Design Optimization,” Mathematical 
Programming, 3, 263-275. 
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cycle length = x3 


length = 


FIGURE 14.5 Cofferdam Example Structure 


Given 
é A total dam length (in feet) 
t A design life of the dam (in days) 
and other characteristics of site and materials, we wish to determine a minimum cost 
design. Main decision variables are 

x; 4 height of the dam (in feet) 

X2 4 average thickness of the dam (in feet) 

x3 A length of a cycle of the dam (in feet) 


Cofferdam Example Model 
To develop a model, we begin with the various elements of cost. Filling cost is roughh 
proportional to the dam volume. Using a cost of $0.21 per cubic foot, this produces 
fill cost ~ 0.21(length) (height) (thickness) 
= 0.21£x1 x2 
Similarly, steel cost depends on the area of the dam’s front and back, plus that of the 
two cylinder sides passing through the dam in each cycle. Pricing steel at $2.28 per 


square foot, we have 


length 
steel cost © 2.28 [2(lengthy(heignt) 2( Bae ) (height) (thickness) 
cycle length 


= 4.560x) + 4.56¢—— 
1B 
A very low dam would minimize construction cost, but flood risk must also be 
considered. Analysis of prior experience suggests that flood cost can be approx- 
mated as 
design life 
flood cost ~ (cost per flood) (ear) 
empirical 


= 40,000~ 
X4 


— 
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where each flood costs $40,000, and x4 is an intermediate decision variable related 
to dam height by 


X4+ 33.3 < 0.8x1 
Other constraints of the model come from possible failure modes, One possi- 
bility is slipping on the river bottom. This can be prevented if 
1.0425(height) < thickness or 1.0425x, <x 
The other main consideration is tension stresses at cycle joints. These require that 
(height)(cycle length) < 2857 or x ,x3 < 2857 
Collecting all the above, simplifying to make all constraint right-hand sides 


= 1, and assuming a dam of length ¢ = 800 feet that should last t = 365 days, our 
cofferdam design task reduces to the following constrained NLP model: 


X1X. 1.46107 
+ See 


min 168x,x2 + 3648x, + 3648—— (cost) 
X3 X4 
cos) 1.6 
s.t. Seuss + a sil (empirical) 
a “1 (14.11) 
1.0425 
mney =i (slipping) 
x2 
0.00035x1.x3 <1 (tension) 


X41, X2,%3,X4 > 0 
An optimal design has height xf = 62.65 feet, average thickness x3 = 65.32 feet, 
cycle length x3 = 45.60 feet, intermediate variable xj = 16.82, and total cost $2.111 
million. 


Posynomial Geometric Programs 


The objective and constraint functions of cofferdam model (14.11) have a special 
posynomial form. 


p(x) is a posynomial if it can be expressed 


0 and exponents a,.; of arbitrary gn. 


_sor ven dk» Oand exponents ay 


For example, objective function 


1.46 x 107 
floc, x2, x3, 4) 2 168x1x9 + 3648x, + 3648 ! — 
3 4 
of model (14.11) is a posynomial with 


d; = 168, d; = 3648, d3 = 3648, ds = 1.46 x 10’ 


Cito ae ha 0 
iV aa 02 aa = 0) a4 0 (14.12) 
ai=1, @32=1, @23=-1, wm4= 0 


a41=0, a42=0, a43= 0, agg=—1 
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Notice that powers a, ; can have any sign, but coefficients d,; must be positive. Thus 
Wer, 2, x3) X13)" a) + 2062) gy 

is a posynomial, but the variation 
h@s xara) 2 8G) Gs) 2902) Ga) 


is not because d, = —29 < 0. 
Posynomial geometric programs are NLPs over posynomial functions and pos- 
itive variables. 


1441 | An NLP isa a posynor 
form 


With posynomial terms written out, the format is 


n 


min oa dk il Ge 


keKo ns 
14.13 
s.t. ae [Lap 21 i=1,...,m ( 
keK; j=1 
xj >0 (Pa ee 


where nonoverlapping the K; index terms k in various posynomials. 

Notice that we allow only a minimize objective function form and < constraints 
Also, we have only positive-valued variables, and positive right-hand sides have been 
divided through to produces 1’s. Usual conversions to reverse directions for other 
cases fail in the geometric programming case because they change the sign of some 
dx and thus destroy the posynomial property. 

Cofferdam model (14.11) satisfies all these conditions. In detail format (14.13). 
model (14.12) shows coefficients for objective function terms in Ko = {1,2,3,4 
Those for constraint sets K; = {5, 6}, Ko = {7}, and K3 = {8} are 


ds =1.25, do = 41.625, d;=1.0425, dg = 0.00035 


a5 = —1, 45,2 = 0, 453 = 0, 45,4 = il 
461 = —l, 16,2 = 0, 163 = 0, 46,4 = 0 
az = ils a2 = —1, a7.3 = 0, a4 = 0 
age else tapos — 20 Rags — deena a— 0 


SAMPLE EXERCISE 14 5 GEOMETRIC PROGRAMS 


Determine whether each of the following NLPs i isa fot connal geometric program 
For those that are, also detail coefficients d; and a,j in standard form (14.13). 


—— #£3 } 
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(a) min 144—— + 6ws 


19w, + (Ww)? < w3 
W1W2W3 < 44 
W1,W2,W3 > 0 


(b) max ae + 6w3 


19w1 — (w2)? < w3 
W1W2W3 > 44 
W1, W2, W3 > 0 


(a) Dividing both constraints by their right-hand sides places this model in posyno- 
mial geometric program format (14.13). Coefficients are 


Ko = {1,2}, Ki = {3,4}, Ka = {5} 

d= 144, d=6, ad=19, d=1, ds= 
aqi= 1, yn2 >= Ds 43> 0) 

a21=0, a= 


0 

0 
@1=1,a@o=°0,  ~“az=-1 

Gai—Onssd4o— 2 

1 


A 


(b) This model is not a posynomial geometric program for several reasons. First, its 
objective maximizes a posynomial, and definition | 14.11 requires a minimize. Simi- 
larly, the second main constraint has > form, not the < as appropriate for geometric 
programs. Finally, the first main constraint function has a negative coefficient, so it 
is not a posynomial. 


Special Tractability of Posynomial Geometric Programs 
Posynomial functions need not be convex, and thus geometric programs/ 14.11 | are 
often not convex programs. For example, 

h(x1, x2) & (x1)?x2 + 7x2 (14.14) 


has Hessian matrix at x = (1, 1) 


Hd.) = (5 nl 


This matrix is not positive semidefinite, and thus (principle | 
over even x > 0. 

Still, posynomial geometric programs can be made convex with a suitable 
change of variables. 


7 |) f is not convex 


‘ams when origi- 
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For example, in the case of function (14.14), substituting z; 4 In(xj) or xj = & 
produces 


h(z1, Z2) 4 (e7)°(e#) + T(e#) = et 4 7(e?) 


Both exponential terms are convex under composition rule " 13.31 |, so their sum is 
also convex, 


tion work. First, a logarithmic edlisfortentn Soil be impossible if any x; could 
be zero or negative. Also, the transformation could produce a mixed- -sign sum Of 


convex terms if all coefficients c, were not positive in definition . Finally, con- 
vex constraint functions would not be appropriate for a convex program (definition 
[14.3]) unless all constraints were of < form. 

Further transformations of posynomial geometric programs can lead to even 
greater tractability. Section 14.9 provides details. 


14,3 LAGRANGE MULTIPLIER METHODS 


If we can see how to view a constrained nonlinear program as one with only = 
constraints (i.e., no inequalities of any form), calculus methods predating most o% 
the numerical search techniques of this book can sometimes be applied to find an 
optimal solution. We briefly explore such Lagrange multiplier techniques in this 
section. Also, Lagrangian ideas will be seen to motivate more general approaches 
in later sections of the chapter. 


Reducing to Equality Form 


Lagrange multiplier solution techniques are most easily applied to models in equality 
constrained format. 


14.13 | Lagrange multiplier solution techniques address NLPs in n pure equality 
form 


St 4 = for all i= 1, I 


That is, they consider on a set of constraints ened active, which can be 
taken as = 'S. 
There is one equality for each original equality and one for each active inequality 
Notice that not even variable-type inequalities such as nonnegativity constraints are 
allowed. 
To see how such models can arise, consider the Pfizer lot sizing model of Sectio= 
14.2: 


: 2160 8712 E 
min f(x) 4 66.21x, + ae + 426.8x2 + —— (total cost) (14.15 
1 X2 


2916 


X4 


+ 61.20x3 + 
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18009) 726 , 227.5. 243 
Se SP ee) (production time) 
x1 X2 x3 X4 


Gila ee tre @reeeel ©) 


If none of the constraints are active at an optimal x*, that solution will be an uncon- 
strained optimum of the objective function alone. Since we have shown in Section 
14.2 that the objective function is convex, we may compute x* by finding a station- 


ary point (principle | 13.22 |) i.e., a point where all partial derivatives = 0. There x} 


solves 
of 2160 2160 
— = 66.21 — =0 hat = == 5): 
ax (1)? Sak 4) Veen = 


Similarly, unconstrained x} = 4.518, x3 = 2.322, and XZ = 3.298. 
Checking the main constraint of (14.15) gives 
180 726 DIES 243 
5712 | 4518 * 2.322 + 3.298 
Thus our unconstrained solution is infeasible, and the main constraint must be treated 
as active at a true optimum. Assuming that nonnegativity constraints will remain 
inactive, we may deal with the model in pure equality form 


Dill 2215 


216 8712 
min f(x) A 66.21x, + = + 426.8x2 + —— (total cost) 
1 
2916 
+ 61.20x3 + = + 268.1x4 + ro16, (14.16) 
X4 

80 726 27.5 243 
st. igh + Jef + ms eS (production time) 

x4 X2 x3 X4 


eosrangian Function and FGM M linliens 


to an unconstrained form by weighting constraints in the objective function with 
Lagrange multipliers, v;. The result is a Lagrangian function of both x and v. 


nction associated wi anonlinear program over erequal 
tS 21 (x) = db, - ) _. 


A 


For example, the Lagrangian function for Pfizer equality form (14.16) is 


2160 8712 330 
L (x1, X2, x3, X4, v) 2 66.21x, + ae + 426.8x2 + Fist + 61.20x3 + ite 
1 2, 5 


2 180 2652S, 4.243 
+ 268.1x%4 + 206 t+yv (2215 — i ) 
X4 XY X2 x3 X4 


(14.17) 
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Notice what happens when we form the Lagrangian. We have relaxed the given 
constrained model into unconstrained form. Still, the Lagrangian function value 
coincides with the original objective function value at every feasible point because 


L(x, v) = fos) + ¥> vi [bi — 81] = f®) + D> vi(0) = foo) (14.18) 
i=1 


p=, 


"SAMPLE EXERCISE 14.7: FORMIN 
Consider the equality-constrained nonnGeAT program 


min 6(x1)? + 4(x2)? + (x3)? 
st. 24x, + 24x. = 360 
Seal 


Form the corresponding Lagrangian function. 


Analysis: Here m = 2. Following format 14.14 |, we form the Lagrangian by rolling 


the 2 equality constraints into the objective function with Lagrange multiplier v; anc 
v2. The result is 


L(x, X2, X3, V1, V2) BS 6(x1)" Ar A(x)? + (x3)? + vy (360 — 24x, — 24x) + v2 (1 — x3) 


Stationary Points of the Lagrangian Function 


Think now about stationary points of the Lagrangian [i.e., points where gradient 
VL(x*, v*) = 0]. Components of gradient VL(x, v) are partial derivatives with 
respect to the x and v parts of a solution, respectively. Setting both = 0 produces the 
key stationary-point conditions. 


Return to the model and Lagrangian function of Sample Exercise 14.7. State the 
corresponding stationary-point conditions on x}, x}, x3, vj, and v3. 


15 | for Lagrangian function 


L (x1, X25 X35 Vi, V2) & 6(a1)? + 4(a2)? + (x3)? + v1 (360 — 24x) — 24x2) + v2 (1 — x5 
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Gradients 
Vix) = (12x1, 8x2, 2x3) 
Vgi(x) = (24, 24, 0) 
V82(x) = (0, 0, 1) 


Thus the required conditions are 


24vi = 124 
24v} = 8x3 
tie 
24x53 -+24x5 = 360 
es =! 


Lagrangian Stationary Points and the Original Model 


The value of Lagrangian functions | 14.14] in solving equality-constrained NLPs 
14.13} becomes apparent upon careful examination of Lagrangian stationary con- 


ditions | 14.15 |. We originally formed the Lagrangian to obtain a relaxed model that 
matches the original objective function at feasible points. For any fixed choice v of 
Lagrange multipliers, an en optimum x* of that relaxed model must be a 


stationary point (principle | 13.19 | 13.19} ). This is exactly what the first part of conditions 


14.15 | demand. 

But the second part of | 14.15 | requires any stationary point of the Lagrangian 
to satisfy all constraints of the original model. Thus if the x* components of a sta- 
tionary point can be shown to optimize the relaxation with multipliers fixed at the v* 
of a stationary point, that x* solves a relaxed form, achieves feasibility, and has the 
same objective value as the original f(x*). It must solve the constrained model. 


(Readers may wish to compare with principle | 12.10] of Section 12.2 for a similar 
result in the context of integer programming.) 


Lagrange Multiplier Procedure 


The Lagrangian approach to solving equality-constrained NLPs exploits sufficient 
optimality condition | 14, 16). Specifically, we: 


1. Reduce the given model to pure equality form| 1 


Form Lagrangian function | 14.14 |. 


2. 

3. Solve conditions| 14.15 | for a stationary point of the Lagrangian function. 

4. ‘Try to establish that the x = of the stationary point is optimal for the Lagrangian with 
v = yv* and thus (principle | 14.16 | 14 16 |) optimal for the original model. 
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Lagrangian (14.17) shows the result of steps 1 and 2 for our Pfizer lot sizing 
example. Solving for a stationary point, we first compute 


aL 2160 180v 2160 — 180v* 
— = 66.21 — ——{ =0 —— 
Ox, Gol Gas Grae 66.21 
ab yay SP PO eg 2 ee 
ares sues (x2)? (2)? EN 426.8 
none (14.19) 
aL 330 27.5v Sys 
— = 61.20 — —— = — 
ies Deep Gal? Pini 61.20 
aL 2916 243v 2916 — 243v* 
— = 268.1 =() i 
ax4 (x4)? zt (x4)? | \ 268.1 
Then setting 

OL 180 2 215,.3243 

— = 221.5 — — f26 =") 

av X41 X2 x3 X4 


and substituting gives 
27.5 243 


180 726 
+ +g + 
2160-180" /8712—726v* [330—275v* [2916 — 243v* 
66.21 426.8 61.20 268.1 


Now taking advantage of the fact that changeover costs in the objective function 
of model (14.16) are multiples (by $12,000 per week) of changeover times, we may 
factor 


22S 


22S = 


1 180 726 27.5 
—— + + 


243 
SS ~ 
J12—v ‘ 180 F Gi acme 7 es om oa OY 
66.21 V4268 V6120 YV 2681 


[ /180(66.21) + /726(426.8) + /27.5(61.20) + V3 268.) 


and solve 


1 
22NES 
—6.865 


Veale) 


Finally, substitution is (14.19) gives xf = 7.161, x} = 5.665, x3 = 2.911, and xj = 
4.135. 

We know that this x* part of the stationary point solution yields an optimum 
because it agrees with values reported in Section 14.2. To complete step 4 of the 
Lagrangian approach and apply principle | 14.16], however, we must examine the 
Lagrangian objective with v = v* = —6.865: 


2160 1D 330 
L (x1, X2, X3, X4, —6.865) A 66.21%, + —— + 426.8x2 + oie + 61.20x3 + — 
xy X2 x3 


291 
4 


180 726 27.5 =) 
1 x2 X3 X4 
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3395.7 13,6 8. 
= 66.21%, + —— + 426.8x) + a6 + 61.20x3 + dsc 
x4 x2 x3 


4584.2 y 
+ 268.1%4 + ag. ors 1520.6 
4 


This function is convex for the same reasons that the original objective was in Section 
14.2. Thus the computed stationary point is indeed a global minimum (principle 


13.22 ]). 


Use terns euuIplier'n raetiods to corEuteTa an sauna ates to the ode &f 
Sample Exercises 14.7 and 14.8. 


Analysis: Stationary-point conditions from Sample Exercise 14.8 are 


24vi = 12x 
24v} = 8x5 
+1y3 = 2x3 
24xi + 24x5 = 360 
1x3 =1 
Using the first 3 to substitute for x;,...,x3 in the last 2 of these constraints yields 
24(2vt) + 24(3vt) = 360 or ve =3 
1(5v3) =1 or ve =2 


Then solving for corresponding x; produces the unique stationary point 
Ons id= y,9, 13,12) 
It remains to be sure that the computed point is a minimum of the Lagrangian function 
L (x1, x2, X3, V}, V3) 
S 6(x1)? + 4(x2)? + (x3)? + 3 (360 — 24xy — 24xp) +2 (1 — x3) 
= 6(x1)? + 4(a2)? + (a3)? — 72x — 72x — 2x3 + 1082 
But this function is obviously convex (squares plus linear), so that the computed 


stationary point does indeed solve the original constrained model (principle| 14.16 ys 


Interpretation of Lagrange Multipliers 
Those readers who have followed large parts of this book will recall that we employed 
dual variables v; on constraints of linear programs (definition | 7.20 |, Section 7.3) to 


analyze the sensitivity of results to changes in the right-hand-side coefficients of the 
model. Itisno accident that the same notation is used here for Lagrange multipliers. 


“can be int rprete 
tight-hand side b;. 
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To see that this interpretation applies, we need only examine the Lagrangian 
function at an optimal (x*, v*): 


L(x*, v*) =f") + de vj [bi — a(x") 


i=1 
The rate of change with RHS 5; is 


aL = Wee 
GD; 
For a specific example, return to the v* = —6.865 on the sole constraint of 


Pfizer equality model (14.16). This quantity is the partial derivative of Lagrangian 
function (14.17) with respect to the right-hand side, which represents a production 
capacity in weeks. Thus because the Lagrangian function and the true objective 
function coincide at stationary points [expression (14.18)], this v* tells us that smal! 
increases in the RHS would decrease (improve) the optimal objective value at the 
rate of $6865 per week. 


Use the optimal Lagrange multipliers of Sample Exercise 14.9 to analyze sensitivity 
of results to changes in constraint right-hand sides. 


Analysis: From Sample Exercise 14.7, model constraints are 

24x, + 24% = 360 

peel 
Corresponding optimal Lagrange multipliers are vj = 3 and v; = 2. We see that an 
increase in either right-hand side would increase (degrade) the optimal value of 541 


At least for small changes, every unit increase in RHS 360 adds vj} = 3, and each unit 
increase in RHS 1 costs v5 = 2. 


Limitations of the Lagrangian Approach 


Although Lagrange multiplier methods work well for some models, it should be 
apparent that they have serious limitations: 


e Stationary-point conditions | 14.15 | can be solved only if they are linear or very simp!< 
nonlinear functions. In other cases, solving those conditions may be more difficult thas 
directly searching for an optimal solution to the original model. 


e Ifa given model has many inequality constraints, it may be an explosively combinatorie 
task to determine which will be active at an optimal solution, so that equality-constraine< 
Lagrangian methods may be applied. 


e Wecan be certain that the stationary point computed from system | 14.15 | is a globs 
optimum only if the original model functions were tractable enough to apply princip’: 


There is also another, more subtle difficulty with applying Lagrange multiplics 
techniques to some NLPs. Principle | 14.16 | tells us when the x* part of an optimum 
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for the Lagrangian function must be optimal in the original model. But the converse 
is not always true. That is, an optimal solution in the original model need not cor- 
respond to a stationary point of the associated Lagrangian. Although most models 


occurring in application do have optima satisfying conditions| 14.15 |, we will present 
an example in the next section where the property fails. 


Cancer the equality- monenained nonlinear program 


max ww 
st.  9(w1)* — 17(1)? + 6(w1)? + 3; + Le” = 100 


Describe the difficulties that would be encountered in trying to address this model 
by Lagrange multiplier methods. 


Analysis: Using Lagrange multiplier v on the single constraint yields Lagrangian 


L(y, Wo, v) A wiw2 + [100 =9@4)t + (wes 6(w1)? — 3w1 — 11e"?] 


Corresponding stationary point conditions| 14.15] are 


OL 

— = WwW? — 36v(w1)? + 51v(w1)? — 12vw1 — 3v = 0) 

aw) 

a = Ww, — 1lve” = 0 

awy 

aL 4 3 2 

Dun = 100 — 9(w1)* + 17091)? — 6(w)? — 3w, — le” = 0 
1 


Solving these highly nonlinear conditions would probably be as difficult as solving 
the original NLP. Furthermore, the original objective function is neither convex nor 
concave, and the constraint function is still less tractable. Even if a stationary point 
could be computed, it would probably be impossible to argue that the W], W5 part 
represented an optimal solution. 


14 4, KARUSH-KUHN-TUCKER OPTIMALITY CONDITIONS 


The Lagrangian stationary-point conditions| 14,15 |are often difficult to solve directly 
for an optimum, but they do provide useful conditions that optima must (usually) 
satisfy. In this section we develop the elaborated form known as Karush-Kuhn- 
Tucker (KKT) conditions, which we will see are intimately related to whether a 


point is a local optimum of the given NLP. 


Full Differentiable NLP Model 


The Lagrangian discussion of Section 14.3 deals only with equality constraints. KKT 
conditions address the full differentiable nonlinear program. 
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i418 | Differentiable nonlinea - programs: ee 


Complementary Slackness Conditions 


The difficulty in extending Lagrangian stationarity conditions | 14.15 | to inequality 
cases arises in knowing what inequalities will be active at a local optimum x (i.e.. 
hold as equality). When we know that an inequality will be active, we may treat it as 
an equality and include it in the Lagrangian. If it will be inactive, we want it left out. 

One way to formalize such requirements is to assign a Lagrange variable v; 
to every constraint but require those for inactive inequalities to = 0. That is, we 
enforce complementary slackness constraints like those of | 7.26| (Section 7.3) for 
linear programs. 


14.19 i Either i ine equality constraints should be active ata local optimu : ‘or the 
La at is 


_ ee Vj (by S oe = 0 ‘fora all inequalities 


We may illustrate with quadratic portfolio example (14.10). 
min 66.51(%)* + 2(2.61)x1x2 + 2(2.18)x1x3 


+ 0.63(x2)? (variance) 
+ 2(0.48)x2x3 + 0.38(x3)? 
St. xy $42 +43 = 1 (invest 100%) (14.20) 
13.22x, + 8.24x7 + 9.03x3 > 11 (return) 


X1,X2,x3 > 0 


Numbering constraints in the order given, the corresponding complementary slack- 
ness conditions are 


v2(11 — 13.22x1 — 8.24x2 — 9.03x3) = 0 


tea Seq (14.21) 
v4(—x2) = 0 ; 
v5(—x3) = 0 


Notice that none is needed on equality constraint i = 1. 


Lagrange Multiplier Sign Restrictions 


We saw with interpretation | 14.17] that Lagrange multipliers should reflect the rate 
of change in optimal value per unit increase in right-hand side b;. Just as with 
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linear programming results | 7.21], this interpretation implies Lagrange multiplier 


sign restrictions when the constraints are inequalities. For example, we know that 
increasing RHS b; relaxes a < inequality, so that it can only increase the optimal 
value in a maximize problem or decrease it in a minimize problem. Other cases are 
similar. 


lowing sign restrictions: 


Objective iiss 


Minimize <0 Ve0 Unrestricted — 
Maximize. v,20. v;<0 Unrestricted 


Again illustrating with minimizing portfolio management model (14.20), the needed 
sign restrictions are 


V1 URS; v2, V3, Va, V5 = 0 (14.22) 
because the first constraint is an equality, and the rest are >’s of a minimize model. 


KKT Conditions and KKT Points 
We are now ready to state Karush—Kuhn-Tucker conditions for general (differentiable) 
model | 14.18}. 


Our portfolio model (14.20) has objective function gradient 


133.02x, + 5.22x2 + 4.36x3 
ViGh, x9, x3) = 5.22x1 + 1.26x2 + 0.96x3 
4.36x1 + 0.96x2 + 0.76x3 


820 Chapter Fourteen e Constrained Nonlinear Programming 


and those of the 5 linear constraints are 
Vg1(%1, x2, x3) = (1,1, 1) 
V 22 (X41, X2, %3) = (13.22, 8.24, 9.03) 
V3 (1, 42, %3) = (1, 0, 0) 
Vga(x1, X2, x3) = (0, 1, 0) 
Vg5(%1, 2,3) = (0,0, 1) 


Thus the gradient equation part of KKT conditions | 14.21 | is 
vy, +:13.22v2 + v3 = 133.02x, + 5.22x7 + 4.36x3 
1yy + 8.24v2 + v4 = 5.22x, + 1.26x2 + 0.96x3 (14.23) 
ly, +. 9.032 + v5 = 4.36x; + 0.96x2 + 0.76x3 
The rest of the conditions are primal constraints 
M+%2+%3=1 
13.22x; + 8.24x2 + 9.03x3 > 11 (14.24) 
X1,X2,x%3 >0 


complementary slackness (14.21), and sign restrictions (14.22). 

Notice the direct parallel to Lagrangian stationary-point conditions [14.15 I. 
Both sets of conditions require the objective function gradient to be expressible as 
a multiplier-weighted combination of constraint gradients, while primal constraints 
are also satisfied. The new elements are complementary slackness conditions and 
sign restrictions arising from inequalities. 


| SAMPLE EXERCISE 14.12: FORMULATING KKT CONDITIONS 
Consider the nonlinear program 


max 2w,+7w2 
st. (wy — 2)? + (w2 -2)% = 1 
wy, <2 
W2 <2 
w, > 0 
Wz >0 


State the Karush-Kuhn—Tucker conditions for this model. 


Analysis: We apply definition | 14. - . Numbering constraints in the order given. 


Vf1,w2) = (2,7) 
Vgi(wi, w2) = (201, 2w2) 
Vg2(wi, w2) = (1,0) 
Vg3(w1, w2) = (0, 1) 
Vga(wi,w2) = (1, 0) 
V85(wi, W2) = (0, 1) 


eee oe 
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Thus the KKT conditions consist of primal constraints 
(w1 — 2)? + 2 — 2) =1 
wy, <2 
Ww, <2 
w, >0 
w2 >0 


gradient equation 


Or Vatae leave) a 


complementary slackness 


v2(2 —w1) = 0 
v3(2 — w2) = 0 
v4(0 — w1) = 0 
v5(0 —W2)= 0 
and sign restrictions 
v2, v3 = 0 
v3, V4 0 


Improving Feasible Directions and Local Optima Revisited 


To see the importance of KKT conditions| 14.21 | in constrained nonlinear program- 
ming, we must return to the elementary improving search notions of Sections 3.2 and 
3.3. Move directions Ax pursued by implementations of improving search should 
be both improving and feasible. That is (definitions and | 3.141), they should 


improve the objective and maintain feasibility for sufficiently small steps i. 
If there is such an improving feasible direction available at a current solution 


in the search, the point cannot be even locally optimal (principle | 3.16 |). Progress is 
still possible in every neighborhood of the current point by advancing in the available 
direction. 

When no improving feasible direction exists, the current solution is under mild 
assumptions, at least a local optimum (principle| 3.17 |). Still, cases like Figure 3.8(b) 
show that the absence of improving feasible directions does not always imply local 
optimality. We can only be certain the search will stop. 


ing search to stop, provide: 
Seen reached = 


When does a direction Ax improve at current x? First-order Taylor series 
approximation (principle | 13.17 |) 
f(x + Ax) © f(x) + Vf(x)-Ax 
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suggests that improvement depends on the sign of Vf(x)- Ax. This yields conditions 
3.21 | and 


1408 The linear Taylor appr ° smoc 


tive function f(x) shows 


Ax>0 Vf (x) Ax <0 
I Nonimproving — 
Improving 


A direction can still improve if Vf(x)-Ax = 0, but the absence of any Ax satisfying 
first-order condition | 14.23 | is a strong indication that no improving directions exist. 
3.25 | are exact for linear constraints 


The corresponding feasibility conditions | 
Direction Ax is feasible if 


<0 for active constraints a-x <b 
a-Ax{ =0 for all constraints a-x=b 
>0 for active constraints a-x>b 


Inactive constraints need not be considered because they have no immediate impact 
on feasibility. 
To generalize for nonlinear =). we may again employ first-order Taylor 
series approximations (principle | 13.17] 87 |): 
gi(x + Ax) © g(x) + Vgi(x)-Ax (14.25 
If g; is active at x, g(x) = b;. Thus feasibility in (14.25) depends on the sign of the 
term Vg;(x)-Ax. 


14.24 | Direction Ax is feasible at x for the linear Taylor approxi ation to 
Strained oo program [14.18 | if 


KKT Conditions and Existence of Improving Feasible Directions 


We are now in a position to link Karush-Kuhn—Tucker conditions | 14.21 | with the 
existence of improving feasible directions at a current search point x. 
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To illustrate, recall that an optimal solution for portfolio management model 
(14.20) is 
7 = 0430.5 SHO. 45 S107 
Certainly, there are no improving feasible directions at this global optimum, and we 
may demonstrate that fact by finding corresponding v; to satisfy KKT conditions 
(14.21) to (14.24). Values that will do the job are 
vz = —132.026, v3 =14.892, v3=0.0, vy=12.275, vi=0.0 (14.26) 
It is easy to check that these primal and multiplier values satisfy complementary 


slackness conditions (14.21), sign restrictions (14.22), and primal constraints (14.24). 
In gradient equation (14.23) 


1yy + 13.22v2 + v3 = 1(—132.026) + 13.22(14.892) + (0.0) = 64.8 
133.02x; + 5.22x2 + 4.36x3 = 133.02(0.47) + 5.22(0) + 4.36(0.53) = 64.8 
ly, + 8.24v2 + v4 = 1(—132.026) + 8.24(14.892) + (12.275) = 2.96 
5.22x1 + 1.26x2 + 0.96x3 = 5.22(0.47)(+1.26(0.0) + 0.96(0.53) = 2.96 
Ly, + 9.03v2 + v5 = 1(—132.026) + 9.03(14.892) + (0.0) = 2.45 
4.36x, + 0.96x2 + 0.76x3 = 4.36(0.47) + 0.96(0.0) + 0.76(0.53) = 2.45 
Contrast this optimal point with x = (1, 0, 0), where direction Ax = (—1, 0, 1) 
satisfies first-order conditions 3 | and because 
Vf, 0, 0)- Ax = (133.02, 5.22, 4.36)- (—1, 0, 1) = —128.66 < 0 

and active constraints 

V2.1, 0,0)-Ax = (1,1, 1)- (—1, 0, 1) =0 

Vega(1, 0,0)-Ax = (0,1, 0)- (—1, 0,1) = 0 

Ve5(1, 0,0)-Ax = (0, 0, 1)- (—1, 0, 1) = 0 


(Note that v2 = v3 = 0 for complementary slackness.) But now the unique solution 
to gradient equation 


1 0 0 
( 1 Jno 1 Js ( 0 Js=( 133.02, 5.22, 4.36 ) 
1 0 1 


yy = 133.02, v2 = —127.8, v3 = —128.66 
which violates sign restrictions v4, v5 > 0. KKT conditions cannot be satisfied. 
To see why principle | 14.25 | must always be true, we may think of improving 


feasible conditions| I and | 14.24 | as a linear program in decision variables Ax. 
Taking the minimize case, and we want to 
min Wf(x)-Ax 
st. WVg(x)-Ax 250 for active >’s 
Vgi(x):Ax <0 for active <’s 
Vegi(x)-Ax =0 for all =’s 


(14.27) 


(14.28) 
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Now we apply some linear programming duality from Section 7.4. Over multipliers 
v;, the dual of (14.28) is 


max ys (O)v; = 0 
i active 


st. > Vgi@ vi = Vix) (14.29) 


i active 
vy; >0 _ for active >’s 


v; <0 for active <’s 
Notice that the feasibility requirements of dual (14.29) are identical to the 
gradient equation and sign restriction part of KKT conditions | 14.21 | at x (assuming 


that v; = 0 on inactive constraints to satisfy complementary slackness). If any v; fulfill 
these conditions, the dual is feasible and its objective value is constant zero. It follows 


(principle ) that the optimal Ax in primal (14.28) has the same optimal value 
Vf(x)-Ax = 0. If KKT conditions are fulfilled, no Ax can fulfill all the conditions 


of and : 

On the other hand, if some Ax meets all the conditions of | 14,23 | and | 14.24). 
there is a feasible solution to the primal with Vf(x)- Ax < 0. Then the dual must be 
infeasible, because every dual solution bounds the primal optimum (principle 


and any would have objective value = 0. It follows that KKT conditions cannot be 
fulfilled at x. 


Consider the constrained NLP 
min (w,)? + (w2)? 
S.t. wi tw2=1 
W1,W2 >0 


1 


A global optimum is w} = w3 = me 


(a) State the KKT conditions for this problem. 


(b) Verify that Aw = (1, —1) satisfies the first-order conditions for an improving 
feasible direction at w = (0, 1), and that the corresponding KKT conditions have no 
solution. 


(c) Verity that KKT conditions hold at the optimal w*, so that no direction can meet 
first-order tests for being improving and feasible. 


Analysis: 


(a) Following | 14.21) with Lagrange multipliers v, v2, v3 on the three constraints. 
conditions are 


wy+w2z=1 (primal constraints) 

W1,W2 > 0 

v2(—w1) = 0 (complementary slackness) 
v¥3(—w2) = 0 
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( ; ) Viet ( : ) v2 + ( : ) y3= ( ae ) (gradient equation) 


v2, ¥3 = 0 (sign restrictions) 


VfO, 1)-Aw = (0, 2)- (1, -1) < 0 


It is also feasible because active constraints have 
V21(0, 1) -Aw = (1, 1): @ ee) — 0 
V292(0, 1) -Aw = (1, 0): Gs eal) = 0 


Solution w = (0, 1) satisfies the primal constraint part of KKT conditions in 
part (a), and making v2. = 0 will assure complementary slackness. Solving gradient 


equation 
1 1 0 
Goals Caley, 


yields unique solution v; = 2, v2 = —2, which violates the sign restriction on Vv». 
KKT conditions cannot be satisfied. 


(c) Optimum wy = w5 = 5 satisfies primal constraints and is active only in the first, 
thus corresponding vz = v3 = 0 to conform to complementary slackness. This leaves 


gradient equation 
1 ey frl 
fairer \ dl 


which has solution v; = 1 satisfying all sign restrictions. KKT conditions do hold. 


Sufficiency of KKT Conditions for Optimality 


Since principle 14.25 shows that a KKT point is one that admits no direction satisfy- 


ing first-order conditions for improving feasibility, it follows that KKT conditions are 
sufficient to establish optimality whenever the absence of improving feasible direc- 


tions is sufficient. The most common case is convex programs (definition i 


ke 


For example, our portfolio example (14.20) is a convex program because all its 
constraints are linear, and its objective function is convex because it has positive- 


definite Hessian matrix 
133.02 5.22 4.36 
Hx = 5.22 1.26 0.96 (14.30) 


4.36 0.96 0.76 


Thus, when we verified that xj = 0.47, x5 = 0.0, x} = 0.53 is a KKT point in 
computation (14.27), we proved that the solution was optimal (principle | 14.26 )). 
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Necessity of KKT Conditions for Optimality 


A much more subtle issue than when KKT conditions are sufficient to establish a 
point’s optimality is when they are necessary. That is, when must optimal points 
satisfy KKT conditions? 

To see the issue, consider the NLP 


min (y1)? + 4y2 
st. (y¥,.-1)? +()? <1 (14.31) 
(i+)? +(02)? <1 


It is easy to check that this model is a convex program because the objective and 
both constraints are convex. Also, the only feasible solution y; = y2 = 0 has to be 
optimal. 


For KKT conditions| 14.21. 
Vf15 y2) = 291.4), Veit, y2) = 2y1 —2,2y2), Veo01, y2) = (291 +2, 2yr) 


so that the gradient equation part at y = (0, 0) becomes 


(o)»+(o)e=(4) 


Clearly, there is no solution v;, v2. Even though y; = y2 = 0 is a global optimum of 
a convex program, it is not a KKT point. 

Fortunately, such cases where KKT conditions are not necessary are rare in 
common models. Also, a variety of constraint qualifications have been derived to 
characterize models where every local or global optimum is a KKT point. We present 
here only the easiest to apply. 


14. == aS local optimal solution ofa constrained differentiable As must be a 


Without actually cae verify that every ioeal optimum ot the foilowilza models 
must be a KKT point. 


(a) max (w1)?+e”+wiw 
s.t. 3wy +w2 <9 
W1,W2 > 0 


(b) max (1)? +e" + wiw2 


s.t. (Ww — 1)? <1 
(w2 —2)? <4 


Analysis: We apply constraint qualifications| 14. 27, which depend only on the con- 
straints of the models. 


(a) All constraints of this model are linear, so every local optimum must be a KKT 
point. 
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(b) Constraint gradients are 
Vg1(W1, W2) = (2w1 — 2,0) and WVg2(w1, w2) = (0, 2w2 — 4) 
These constraints are linearly independent except at w = (1, 2), which is not feasible. 


Thus the active constraints at any local optimum will be linearly independent, and 
all such solutions must satisfy KKT conditions. 


14,5 PENALTY AND BARRIERMETHODS 
One approach to solving constrained nonlinear programs is to convert them to a series 


of unconstrained ones. In this section we investigate such sequential unconstrained 
min/maximization techniques (SUMT), also known as penalty and barrier methods. 


Penalty Methods 


One scheme for transforming constrained into unconstrained NLPs uses penalty 
methods. 


Many alternatives are available for the penalty functions p;(x) associated with par- 
ticular constraints. 


and max?{0, b; — gi(%)] for =’s 
= max (0, g(x) — = By for <5 
|gi(x) — B,|* for =’s 


Each imposes no penalty when the corresponding constraint is satisfied, but adds a 
growing cost if it is violated. 


EXAMPLE 14.7: SERVICE DESK DESIGN 


Penalty methods are most often used in engineering design applications where many 
of the constraints are nonlinear. We will illustrate with a contrived example to design 
the service desk of a catalog order company. 

Figure 14.6 displays the problem. A service desk 5 meter in width is to be 
centered around two 1-meter conveyors bringing orders from warehouse storage. 
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10-m limit 
service desk 


aes Shes a 
o= ald 


conveyors 


FIGURE 14.6 Service Desk Design Example 


The conveyors are 6 meters apart (center-to-center) and protrude 0.75 meter into 
the work area. For employees to work efficiently behind the counter, there should be 
at least 2 meters clearance in front of the conveyors, and no part of the inside counter 
perimeter should total more than 10 meters from the conveyors. Within these limits 
we wish to maximize the customer room provided by the outside perimeter of the 
counter. 

To model this simple example, introduce an origin halfway between the conveyors, 
and define decision variables 


x1 4 half-length of the work area inside the counter 


x2 A width of the work area inside the counter 


Then the problem can be modeled: 


max 2x; +2x, +2 (outer perimeter) 
2 a 
s.t. se =e <1 (10-m distance limit) (14.32) 
x, = 3.5 (outside conveyors) 
X2 > 2.75 (2-m inside space) 


The objective function maximizes the outer perimeter. The first (ellipse) constraint 
keeps the most distant point inside the counter at most a total of 10 meters from 
the conveyors. The lower bound on x, ensures that the counter falls outside the 
conveyors, and that of x2 enforces the 2-meter inside clearance. An optimal design 
uses a desk with inside dimensions 2x} = 2(3.63) meters by x} = 2.75 meters and 
outside perimeter 14.76 meters. 


Penalty Treatment of the Service Desk Example 


Any of the penalty function alternatives in construction] 14.29 | might be used to deal 


with the < constraint and two > constraints of service desk model (14.32). We will 
choose the second, squared penalty forms. For example, in the first constraint 
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Gx)? @)? 
Pam eaalG, Cs 

When the constraint is satisfied [(x1)*/25 + (x2)?/16 — 1] < 0 and pi(x,x%2) = 0. 
However, violations of the constraint make [(x;)*/25 + (x2)?/16 — 1] > 0 and impose 
a penalty equal to the square of the violation. Proceeding in this manner with all 
constraints yields the unconstrained penalty model 


Ge)? ole 
25 sii allo -1| 


pi(x1, x2) & max* {0 


max 2x,+2x%.+2-—pu (max? {0 
(14.33) 
+ max2{0, 3.5 — x;} + max?{0, 2.75 — =) 


Infeasible solutions in the constrained model (14.32) are now allowed, but they are 
discouraged by subtracting a penalty in the objective function. 


NG PENALTY Moves 


Use absolute value ERNE penalty functions to reduce the following constrained 
NLP to an unconstrained penalty model. 


min (w,)* —wiw2w3 

s.t. Wi t+w2+w3=5 
(w1)? + (W2)? <9 
w3W2 > 1 


Analysis: Using the first, unsquared penalty alternatives of | 14.29 |, the correspond- 
ing unconstrained model | 14.28 | is 


min (w1)* — wiwaw3 + We (|w1 + w2 + w3 — 5| 
+ max{0, (w1)? + (w2)? — 9} + max{0, 1 — w3W}) 


where yu is a positive penalty multiplier. 


Concluding Constrained Optimality with Penalties 


By definition, the penalty terms of constructions os | must = 0 at any x feasible in 
the given constrained NLP. This provides a way to know when unconstrained opti- 


mization of the penalty problem yields an optimal solution for the original model. 


: 14. 30) [fan opti al x* in unconstrained pe 


problem [1 is s feasible in 2 


Any better solution to the constrained model would also have all penalty terms = 0, 
so it would have to best x* in penalty model objective value. 


Differentiability of Penalty Functions 


One consideration in choosing among the penalty options in| 14.29 | is differentiabil- 
ity. Most of Chapter 13’s unconstrained methods that might be employed to optimize 
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penalty model | 14.28 | assume that the function is smooth. None of the first options 


listed in principle | 14.29 | meets this differentiability requirement, but the second, 
squared options do. 


: 9 | are differentiable when- 


We can see why by examining a < inequality g;(x) < b; with g; smooth. The 
corresponding squared penalty term can be expressed 


0 if x satisfies g;(x) < D; 
Lee | [gi(x) — bi} otherwise 
Associated partial derivatives are 
pa! ; if x satisfies g;(x) < b; 
OX; 2[gi(x) — bila 


Notice that these two expressions match at the boundary where g;(x) = b;. Thus 
partial derivatives are well defined and continuous. 


otherwise 


Exact Penalty Functions 
We would also like penalty functions to be exact penalty functions. That is, we would 
like there to be a large enough jz > 0 that the unconstrained penalty model F(x) 
yields an optimal solution in the original constrained form under principle 
driving out all infeasibility. 

A trivial example shows that the squared alternatives of | 14.29 may not be 
exact. Consider 


min y 
Sh y=30 Oe) 

The squared choice in| 14.29 | produces penalty model 
min F(y) & y + ~max7{0, —y} 


fe ify >0 


Via a they 
Differentiating 
dF 1 a yO 
ae ify <0 
shows that the only stationary point is 


a = _ 
Jat i 
But this unconstrained minimum is negative and infeasible, regardless of the magni- 
tude of pw. 
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Suppose that we had used nondifferentiable penalty function max{0, b; — g;(x)} 
on example (14.34). The corresponding penalty model is 


min F(y) 4 y+ wmax{0, —y} 


Now for any > 1, F(y) is minimized at y = 0. That is, a finite yz is large enough to 
make the penalty optimum feasible in the original NLP. 


All of the nonsquared penalty forms in | 14.29] are exact in this way. 


29]: are > applied to a constrained nl 
nd mild ae mplions hold, ther 


Managing the Penalty Multiplier 
In squared cases such as (14.33), a constrained optimum can be obtained with penalty 
methods only by letting ~ — oo. With exact methods of principle 3| there is a 
large enough finite z to do the job. Either way, j. needs to grow large, and we cannot 
know how large when we begin. 

Why not just use a very large , from the start? Figure 14.7 shows the risk with 
trivial model 


min w 
14.35 
Sita oo = WS ( ) 
100 100 
80 80 
60 60 
= = 
a a 
40 40 
20 4 20 
0 as 0 are] 
0 1 2 3 4 5 6 7 8 0 il 2 3 4 5 6 7 8 
w w 
(a) Smaller (b) Larger u 


FIGURE 14.7. Effect of Penalty Multiplier on Unconstrained Tractability 
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When jz is comparitively large, the corresponding penalty objective function F be- 
comes very steep. Small moves have dramatic impacts on its value. The result is 
an unconstrained model that is difficult to solve with any of the methods of Chap- 
ter 13. 

These competing demands on the penalty multiplier , motivate a sequential 
strategy that slowly increases the multiplier. 


Segawa as a Penalty reehntne 


Formalization of principle ’s strategy for slowly increasing the penalty multi- 
plier produces the sequential unconstrained penalty technique of Algorithm 14A. 
Multiplier begins relatively small and grows with each search. For each value of 
#4, unconstrained penalty problem is solved beginning with the optimum of 
the preceding search. If the result is ever feasible in the original model, we stop with 
an optimum (principle [14.30]). Otherwise, we continue until the unconstrained 
optimum is sufficiently close to feasible. 


ALGORITHM 144: SEQUENTIAL UNCONSTRAINED PENALTY TECHNIQUE — 


Step 0: Initialization. Form penalty model [14.28], and choose initial 
penalty multipier jzo>0 relatively small and starting solution x®. Also, 
initialize solution index t <— 0, and pick an escalation factor 6 > 1. 

Step 1: Unconstrained Optimization. Beginning from x, solve penalty 


optimization problem | 14.28 | with 1 = 2; to produce optimum x“), 
Step 2: Stopping. If xt! is feasible or sufficiently close to feasible in the 
constrained model given, stop and output x“*!). 
Step 3: Increase. Enlarge the penalty multiplier as 


Hi+1 <— Buy 
Then advance t <— t+ 1, and return to Step 1. 


Table 14.3 illustrates Algorithm 14A for service desk design model (14.32) and 
corresponding penalty form (14.33). With initial multiplier ~ = 5 a first search 
produces the unconstrained optimum 


x = (9,690, 6.202) 


which violates the first constraint of the original model by 5.160. Then the multiplier 
is increased by factor 6 = 4 and a new unconstrained search initiated from x’. 
The resulting optimum x starts a third search after jz is quadrupled again. The 
process continues until x is large enough that the unconstrained optimum approaches 
feasibility. There we stop to obtain constrained optimal solution x* = (3.63, 2.75). 


Service Desk Example 


Optimal 
t mM x@+D 


0 ; (9.690, 6.202) 
1 1 (6.632, 4.244) 
2 4 (4.981, 3.188) 
3 16 (4.221, 2.749) 
4 64. (3.806, 2.748) 
5 256 (3.677, 2.749) 
6 1024 (3.643, 2.750) 
7 4096 (3.634, 2.750) 


Barrier Methods 
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TABLE 14.3 Sequential Penalty Solution of 


a _—_———— 


Constraint Violation, i 
1 2 3 
5.160 0.000 0.000 
1.885 0.000 0.000 
0.627 0.000 0.000 
0.185 0.000 0.001 
0.051 0.000 0.002 
0.013 0.000 0.001 
0.003 0.000 0.000 
0.001 0.000 0.000 


The penalty methods above begin anywhere and try to force the unconstrained op- 
timum into the feasible set of the given NLP. Barrier methods adopt the alternative 
of beginning with a feasible solution and trying to prevent the unconstrained search 


from leaving the feasible region. 


Since the boundary cannot be avoided with equality constraints, barrier meth- 
ods are applicable only when constraints are all inequalities. Many alternatives are 
available for the g;(x) associated with such constraints. 


Each explodes toward +00 as the corresponding inequality approaches being satisfied 
as an equality. 
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Barrier Treatment of Service Desk Example 


All constraints of service desk design model (14.32) are inequalities, so barrier meth- 
ods could be applied. We will adapt the more common logarithmic forms. For 
example, the first constraint produces barrier term 


(x1)? “@)" 
25) 16 
When x is well inside the feasible region, this barrier function affects the objective 
only modestly. But as ((x1)?/25 + (x2)*/16) > 1, the negative of the logarithm goes 
to +oo. Similar treatment of all constraints yields the unconstrained barrier model 
(x1) 20a)? 
25 16 


qi(%1,%2) £ —In (1 


max 2x, +2+2+u/In(1 — 
(14.36 


+ In(@x — 3.5) + In(x2 - 2.75) 


With jz > 0, an approach to any part of the boundary is discouraged. 


SAMPLE EXERCISE 14.16: FORMING BARRIER MODELS © 


Use reciprocal barrier functions to reduce the following saatioien NLP to an un- 
constrained barrier model. 
min (wy ye —W1W2W3 
st. (wy)? + (W2)? < 9 
W3W2 > 1 
Analysis: Using second, reciprocal barrier alternatives of , the corresponding 
unconstrained model is 


. 1 4 
min (w1)* — wiwaw3 + bw (; — (wy)? — (wo)? a W3W2 — 3) 


where y is a positive barrier multiplier. 


Converging to Optimality with Barrier Methods 


Unlike penalty methods, barrier functions affect the objective function at feasible 
points. We can illustrate the difficulty this causes by returning to the trivial min y, st 


y = 0 example of (14.34). Using, say, the logarithmic barrier alternative of | 14.36 
the corresponding barrier problem is 


min F(y) 4 y— wIniy) 

Differentiating (with y > 0) yields 

dF 

alee lh 

dy y 
which has its only stationary point at 

y=E 

This unconstrained optimum never reaches the true optimum of y = 0 for any pz > 


14.5 Penalty and Barrier Methods 835 


Similar behavior occurs for all barrier versions of constrained NLPs with an optimum 
on the boundary of the feasible set. 


As with penalty methods, however, there is a pattern to the unconstrained 
optima. As uw — 0, the unconstrained optimum comes closer and closer to the 
constrained solution. 


14 38 | Altho 


ean solution! trained NLP as multiplier pp > 0. 


Managing the Barrier Multiplier 


Property | 14.38 | makes it clear that we have to let barrier multipliers 4 approach 
zero if we expect to obtain a constrained optimum. Why not simply start close to 
zero? Figure 14.8 illustrates the difficulty with the trivial model (14.35). 


25 25 

20 20 
~15 ois 
& = 
Q a 

10 10 


nn 
io 
a 


0 0 
3 32 34 36 38 4 42 44 46 48 5 3 32 34 36 38 4 #42 44 46 48 5 
w w 
(a) Larger u (b) Smaller uw 


FicurE 14.8 Effect of Barrier Multiplier on Unconstrained Tractability 


When y is comparatively large, a strong barrier keeps the search far from 
boundary areas, where it might bog down. But small jz’s allow the search to approach 
the boundary. 

The best strategy is to start big and reduce the multiplier slowly. 


igh value =o and decreased 
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Sequential Unconstrained Barrier Technique 


Formalization of principle *s strategy for slowly decreasing the barrier multi- 
plier ,z produces the sequential unconstrained barrier technique of Algorithm 14B. 
Processing begins at an interior feasible x© where none of the constraints are active. 
Multiplier jz starts relatively large and becomes smaller with each search. For each 
value jz, unconstrained barrier problem is solved beginning with the optimum 
of the preceding search. We stop when w is sufficiently close to zero. (Readers may 
wish to compare with barrier methods for linear programming in Section 6.4.) 


| UE 


Step 0: Initialization. Form barrier function | 14.385], and choose initial 
barrier multipier 9 > 0 relatively large and feasible interior starting solution 
x. Also initialize solution index ¢ — 0, and pick a reduction factor B < 1. 

Step 1: Unconstrained Optimization. Beginning from x, solve barrier 


optimization problem | 14.35 | with jz = ju; to produce optimum x@+), 


Step 2: Stopping. If 1 is sufficiently small, stop and output x“+)), 
Step 3: Reduce. Decrease the penalty multiplier as 


Mev — But 
Then advance t <—¢+1, and return to Step 1. 


Table 14.4 illustrates this for service desk design model (14.32) and correspond- 
ing barrier form (14.36). Unlike penalty methods, which can start anywhere, a barrier 
search must begin at an interior feasible point of the constrained NLP. Here we em- 
ploy 

x) = (3.52,2.77) 


The bulk of the computation involves a sequence of unconstrained barrier 
model searches with decreasing multipliers 4. Table 14.4 shows the first used up = 4 
to obtain the unconstrained optimum 


x = (3.573, 2.794) 


Then the multiplier was decreased by a factor of B = : and a new search initiated 
from x. The resulting x starts a third search after yz is halved again. Searches 
continue until yw is close enough to zero that an approximate optimum is at hand. 
Then we stop with constrained optimum x* = (3.63, 2.75). 


TABLE 14.4 Sequential Barrier Solution 
of Service Desk Example 


(3.573, 2.794) 
(3.588, 2.786) 


(3.608, 2.769) 
(3.629, 2.752) 
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14.6 REDUCED GRADIENT ALGORITHMS 


Chapter 5’s simplex algorithm for linear programming is the most widely employed 
of all optimization procedures. In this section we develop natural extensions to 
the nonlinear case known as reduced gradient algorithms, or more generally as the 
variable elimination method. 


Standard Form for NLPs with Linear Constraints 


Most of our development of reduced gradient methods will assume a linearly con- 
strained nonlinear program. In particular, we assume that constraints have the stan- 


dard form like Section 5.1’s definition : 


14.40 


it ed nonlinear programsis 
max f(x) 
Ax ah 


We will also assume for simplicity that the rows of matrix A are linearly independent, 
which can always be achieved by dropping redundant constraints. 
EXAMPLE 14.8: FILTER TUNING 


For a tiny example on which to illustrate reduced gradient notions, consider the 
tuning of an electronic filter. Two parameters 


x, A value of the first tuning parameter 
x2 A value of the second tuning parameter 
must be chosen to minimize distortion 
fier, x2) & (aw — 5)? = 2xyx2 + (2 — 10)? 


with x; in the range [0,3], x2 in the range [0,5], and their total at most 6. The result 
is the linearly constrained nonlinear program 


min f(X1, X2) x (x4 5)? 2x1 xX2 + (x2 10)? 
st. x, +x2 <6 

0O<x <3 

O<x%<5 


(14.37) 


Figure 14.9 graphs this tuning model. A global optimum occurs at x® = 
(1.75, 4.25). 
We may convert to standard form | 14.40 | by adding slacks x3, x4, and xs: 


min f(x1,%2) A (x1 — 5)? — 2x1x2 + (x2 — 10)? 


Siz = -Pxy = Xone =6 
+X] + x4 =) (14.38) 
+X, +x5 =5 


X1,X2,%X3,X4,X5 = 0 
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FIGURE 14.9 Reduced Gradient Search of Filter Tuning Example 


Conditions for Feasible Directions with Linear Constraints 

Any improving search algorithm attempts to move along improving feasible direc- 
tions. Principle [325 | of Section 3.3 has already detailed the requirements for main- 
taining feasibility with standard-form linear constraints. 


i441 Atx feasible for standard-form linear constraints 14.40], Axisa feasible 
direction if 
Axj>=0 forall jwiths, =0_ 


To illustrate, consider point x at (2.5, 0) of tuning example Figure 14.9. In- 
cluding corresponding values for slacks, the full standard-form solution is 


x0 25,0) 357055) 
The only active inequality is the nonnegativity constraint on x2. Thus the correspond- 
ing conditions| 14.41 | for a feasible direction Ax are 


+ Ax, + Axo + Ax3 — (0) 
+ Ax + Ax = 
‘ (14.39) 
+ Axo + Axs = 0 
Ax) > 0 


Bases of the Main Linear Equalities 


Section 5.2 developed the idea of bases or basic column sets of the matrix A. Bases 
are maximal sets of linearly independent columns. 

The important characteristic of a basis, and the corresponding basic variables. 
is that we may solve for the values of basic variables once all other, nonbasic variables 
have been fixed. That is, we may view the basics as functions of the nonbasics. 
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a components. a 


Basic, Nonbasic, and Superbasic Variables 


The simplex algorithms of Chapter 5 proceed through basic solutions in which all 
nonbasic variables take on lower bound value zero. We saw in Section 5.2 how this 
restriction can lead to a search through extreme points of the feasible set. 

With nonlinear models, we know that an optimal solution may very well fall in 
the interior of the feasible set, or at a nonextreme point of the boundary. This does 
not change the fact that basic variables are dependent on our choice of nonbasics. It 
only means that nonbasics cannot be restricted to = 0. A new superbasic category 
of variables arises, which are nonbasics at positive value. 


To illustrate, return to tuning example standard form (14.38), and choose x1, 
x3, and xs basic. Initial solution 


x = (2.5, 0, 3.5, 0.5, 5) 


of Figure 14.9 is implied by nonbasic values x = = 0.0 and xO = = 0.5. Setting all 


nonbasics = 0 would produce an extreme point of the feasible set. But with ne 10, 
and thus superbasic, nonextreme x© can be represented. 


AM E EXERCISE 14.17: DIstING UISHING BASIC, NONBASIC, AN ) SUPERBASIC 


Consider the standard-form nonlinear program 


max f(w) £50 — (w,)? + 6w, — (w2)? + 6w2 + W3 
s.t. + wy —W2 +3W3 = il 

+3w, +2w2 +2w4 = 6 

W1, W2,W3, Wa = 0 


(a) Show that w3 and w, form a basic set of variables. 

(b) Assuming the basis of part (a), classify remaining variables as nonbasic at lower 
bound or superbasic for the solution w = (0, 2,1, 1). 

Analysis: 


(a) A basis is a maximal set of linearly independent vectors. The columns of w3 and 
wg are linearly independent because the corresponding matrix 


B=(5 2) 
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has nonzero determinant = 6, and two is the maximum number of linearly indepen- 
dent 2-vectors. 


(b) The specified solution has w; nonbasic at value zero, and w2 superbasic because 
its value is positive. 


Maintaining Equalities by Solving Main Constraints for Basic 
Variables 


Return now to the feasibility requirements of | 14.41}. Let B be a submatrix of A 


formed by a basic set of columns, and N the submatrix of all other columns. Then 
divide direction vector Ax into corresponding parts denoted Ax™ for components 
on columns in B and Ax for those in N. 


Conditions require that 
AAx = BAx® + NAx™ =0 
Solving for basic components as a function of nonbasic variables, 
Ax® = —B'NAx™ (14.40) 


Thus we can find a direction maintaining the equalities by choosing directional com- 
ponents for nonbasic variables and solving for basics as in (14.40). 


equality constraints es b ie 
Ax® = eo 


where Bi isa basis submatrix of A 4 (B, N). 


For example, solving feasibility conditions (14.39) of our tuning example for 
basic variable components Ax;, Ax2, and Ax3 produces 


Ax} 0) 1 
FO ees PP (cea! (14.41) 


SAMPLE EXERCISE 14.18: SOLVING FOR BAsie 


Return to the maximizing nonlinear program ‘ot Sample E Precise 14. 17 and express 
basic components of a move direction at w = (0, 2, 1, 1) as a function of the nonbasic 
components so that the resulting direction is feasible for the main constraints. 


Analysis: We apply principle | 14.44 | in solving AA w = 0 conditions for basic com- 
ponents. The result is 


Aw3 = —( Aw, — 5 Aw?) 
Aw4 = -(3 Aw, + 1 Aw2) 


14.6 Reduced Gradient Algorithms 


Active Nonnegativities and Degeneracy 


The second part of feasible direction conditions requires that Ax; > 0 when- 
ever the corresponding nonnegativity constraint x; > 01 is active. As with the simplex 
algorithm (see Section 5.6), we will not strictly enforce these requirements on basic j. 
Algorithms developed below endeavor to keep basic variables strictly positive, that 
is, 


xj>0 foralljeB (14.42) 
For example, basic components j = 1, 3, 5 of tuning example solution x = (2.5, 0, 
3.5, 0.5, 5) are all positive. 


Under this nondegeneracy assumption, only nonnegativity constraints for non- 
basics can be active, and we enforce 


Axyp,> 0 foralljeN.. with x =0 (14.43) 


Reduced Gradients 


We now know from construction how to produce a feasible direction by 
limiting independent choices to components for a nonbasic set of variables. It remains 
to construct a feasible direction that improves the objective. 
For small steps the change in objective along direction Ax is the first-order 
Taylor approximation term 
Vi(x)-Ax 


Subdividing the gradient V/(x) into basic and nonbasic parts (V(x), Vf(x)™), 
we can eliminate the basic components to see fully the impact of choices for the 
nonbasics: 


Vi(x)-Ax = Vix) -Ax® + VK) -Ax™ 
~ = Vf(x)®~ (-B-INAx™) + Vf@)™ -Ax™ (14.44) 
= (Vfa)™ — Vf) B-!N) Ax™ 


These derived coefficients on directional components are called the reduced 
gradient. 


‘14.45, | The reduced gradient associated with basis ) 
xisr 4?) ™) with : 
2) - 
{Ma : fi : 


To illustrate, return to tuning model (14.38) at x = (2.5, 0, 3.5, 0.5, 5). The 
corresponding gradient is 


2(x1 — 5) — 2x2 —5 

= 4 2Gn 10) ~25 

Vix) = 0 = 0 
0 0 


0 0 
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Now using system (14.41), which derives B-'N to express feasible direction condi- 
tions with x1, x3, and xs basic, 


r®) = (7,73, 7s) = (0, 0,0) (14.45) 
and 
r™) = (7, 74) 
= Vf™ — vf (B-N) 
De oat (14.46) 
il 0 


= (—25,5) 


SAMPLE EXERCISE 14.19: ComPuTING REDUCED GRADIENTS 


Sas to the nonlinear program of Sample Exercises 14.17 wu 14. 18 wat Based 
= {3, 4}. Compute the corresponding reduced gradient at solution w = (0, 2, 1, 1). 


Analysis: With objective function 
fiw) 50 — (1)? + 61 — (W2)? + Ow. + Ws 
the gradient at the w specified is 


—2w, +6 6 

—2 6 2 

Cie eo tae 
0) 0 


Now applying definition , basic components of the reduced gradient become 
r=1m4=0 
Corresponding nonbasic components are 
r= 6 —(1,0)-(, 5) 
rm =2 —-(,0)-(-},1) 


BIN ws 


Reduced Gradient Move Direction 


Reduced gradient algorithms seek to move nonbasics in reduced gradient direction 
Ax™) = +r) (+4 for maximize models, — for minimize models). However, some 
adjustment must be made to avoid decreasing any x; already = 0 [i-e., to enforce 
feasibility requirement (14.43)]. 


14.6 Reduced Gradient Algorithms 


for basics. 


In the minimizing tuning example of (14.45)-(14.46), construction 
nonbasic components 


Ax, =—%m=25 and Axyg=—rg=—-5 (14.47) 


Corresponding basic components are derived from expression (14.41) as 


AXx4 0 il 95 5 
A =-{1 -1 30 14.48 
(| (: i) el 


We have constructed direction A x of | 14.46 | to be feasible (assuming all basics 
positive). IfAx = 0, the current x can be shown to be a KKT point, and the algorithm 


stops. Otherwise [using (14.44) and| 14.45 |], 
Vix) -Ax = 1) -Ax™ 


SS ery 


Axj=7r;40 


shows that Ax has the proper sign to be an improving direction. 


NSTRUCTING REDUCED GRADIE 


Return to the Henna? program of Sample Exercises 14.17 to 14. 19, BASIE the 
move direction that would be pursued by the reduced gradient algorithm at solution 
= (0, 2,1, 1). 


Analysis: Using reduced gradient results of Sample Exercise 14.19 and construction 


14.46], nonbasic components for this maximizing model are 

Aw; = +r, = & and = Aw2,=4nm=4 
Then basic components derive from the representation of Sample Exercise 14.18 as 
Aw3 = —(3 Aw, — § Aw2) = —2 


Aw, = -@ Aw, +1 Aw?) = — 


Line Search in Reduced Gradient Methods 


Having specified reduced gradient direction | 14.46|, we next need to decide how 
far to follow it. As with most nonlinear methods, a line search will be required 
to determine the largest A for which the direction improves. However, constraints 
add feasibility considerations. Our direction Ax will satisfy equality constraints 
A(x + AAx) = b for arbitrarily large A, but nonnegativity constraints cannot be 


843 


844 


Chapter Fourteen e Constrained Nonlinear Programming 


ignored. The line search must be limited by the same sort of “minimum ratio” check 
employed in LP algorithms of Chapters 5 and 6. 


where x is the current poit 
feasible step 


For example, tuning example direction (14.47)—(14.48) is negative for compo- 
nents j = 3, 4,5. Thus the maximum feasible step at x = (2.5, 0, 3.5, 0.5, 5) is 


Amax = min {33, 3,2} =0.1 (14.49 


30° 5° 25 
Distortion function f(x + Ax) decreases for all A € [0, 0.1], so that the step size 
chosen will be the full A = 0.1. 


Sample Exercise 14.20 computed reduced gradient move direction 


Aw=(2,7,-2,-8) atsolution w= (0,2, 1,1) 


of a standard-form, linearly constrained NLP. Determine the maximum feasible step 
in this direction. 


Analysis: For a standard-form model with linear equality main constraints, the only 
possible loss of feasibility occurs when some variable drops to its lower bound of ' 


This occurs here at (principle | 14.47 |) 


Amax = min | 


dee kal \_5 
es 


10/9’ 65/6) 65 


Basis Changes in Reduced Gradient Methods 


One final issue relates to nondegeneracy assumption (14.42). All our analysis has 
been based on basic variables always having positive values. As with the simple» 
algorithms of Chapter 5 (see Section 5.6), such nondegeneracy cannot always be 
guaranteed. Still, it is sufficient for functioning of the reduced gradient algorithm 
that we replace a variable in the basis if the most recent move forced it to = 0. 
With many nonbasics (and superbasics) changing during the move, it is not as 
easy as with simplex to decide which nonbasic should enter the basis. To be assure 
of keeping a basis, we need only be careful to select a nonbasic actually affecting the 


blocking basic in computation 


14.6 Reduced Gradient Algorithms 


Superbasics are preferred because they have the positive value desired for a basic. 
In the move along tuning example direction (14.47)—-(14.48), the blocking vari- 
able of step computation (14.49) was nonbasic x4. No basis adjustment is required. 


SAMPLE EXERCISE 14. 22: CHANGING THE ASIS IN REDUCED GRADIENT _ : 


The nonlinear Progam of Sample Exercises 14.17 to 14.21 computed maximum fea- 
sible step size Amax = = in the direction 


Aw= q, i, —%. —8) atsolution w= (0,2,1,1) 


with w3 and w4 nonbasic. Assume that a full step 4 = = is chosen by the line search. 
(a) Determine whether a basis change is now needed. 


(b) If a change is required, select a new basis. 


Analysis: We apply principle | 14.48 |. 


(a) After a full step, the new solution will be 


wt dAw = (=, Se 39) 0) 


Since basic variable w4 drops to 0, a basis change is required. 


(b) We must replace w4 in the basis with a nonbasic variable that influenced its value 
on this move—preferably one that is now superbasic. Reference back to Sample 
Exercise 14.20 show that Aw, was affected (had nonzero coefficients) by both non- 
basics. With both nonbasics now superbasic, we arbitrarily choose w2 to produce 
new basis {w2, w3}. 


Reduced Gradient Algorithm 


All the building blocks of a reduced gradient search are now in place. Algorithm 
14C provides details. 

Computation begins at any feasible point and a corresponding basis. Each 
iteration follows reduced gradient direction | 14.46 | until either objective progress 
stops or the feasibility limit is reached. Bases are changes as in | 14.48} whenever 
a basic variable dropping to zero blocks progress. Termination occurs when the 
computed direction is sufficiently close to the zero vector. 


Reduced Gradient Search of Filter Tuning Example 


Figure 14.9 has already displayed the sequence of points visited by reduced gradient 
Algorithm 14C in solving our tuning example from initial point (2.5, 0). Table 14.5 
provides details. 
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Step 0: Initialization. Choose stopping tolerance € > 0 and any starting 
feasible solution x. Then construct a corresponding basis B with as many 
basic x) > 0 as possible, and set solution index ¢ < 0. 

Step 1: Reduced Gradient Direction. Compute reduced gradient r at x 
as in| 14.45 |, and use r to generate move direction Ax’*! per | 14.46). 


Step 2: Stopping. If ||Ax‘''|| < €, stop and output local optimum x. 

Step 3: Feasibility Limit. Compute feasiblity limiting step Amax according 
to | 14.47) (Amax = 00 if Ax*! > 0). 

Step 4: Line Search. Perform a 1-dimensional optimization to determine 
Arai solving 


min or max f(x + aAx'*!) 
s.t. O <2) <Amax 


Step 5: New Point. Advance 


xf) gs x ae deg Axitt 


Step 6: Basis Change. If any basic x/*” = 0, replace one such j in the 


basis with some superbasic j’. 
Step 7: Advance. Increment ¢ < ¢+ 1, and return to Step 1. 


The first move of the search follows the direction (14.47)-(14.48) for a full step 
Amax = 0.1. No basis change is required because the blocking variable is nonbasic. 
Thus gradient computations are simply repeated to produce new direction 


Ax = (0, 21, —21, 0, —21) 
Once again the direction improves all the way to maximum feasible step Amax = 
0.0238. This time, however, the blocking variable is basic x3. Replacing x3 in the 
basis with superbasic x2 keeps basic variables strictly positive without losing linear 


independence of basic columns. 
Recomputation produces the next move direction, 


Ax = (—10, 10, 0, 10, —10) 
Notice that Ax3 = 0 even though —r; = —20, because decreasing x3 would produce 
immediate infeasibility. 
The maximum feasible step in the chosen Ax is Amax = 0.2. Still, a line search 
over A € (0, 0.2] discovers a minimum at A = 0.125. Thus the search advances only 
to 


x® = (1.75, 4.25, 0, 1.25, 0.75) 


This point proves (at least locally) optimal when Ax = 0 computes as the next search 
direction. 
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TABLE 14.5 Reduced Gradient Search of Filter Tuning Example 


xy Xo x3 X4 x5 
min f(x) (x1 — 5)? —2x4,x + (x9 — 10)? b 
1 1 1 0 0 6 
A 1 0 0 1 0 3 
0 1 0 0 1 5 
t=0 B N B N B 
x0) AS 0.0 3.5 0.5 5.0 f(x) = 106.25 
VA)  =-5.0  -25.0 0.0 0.0 0.0 
r 0.0 -25.0 0.0 5.0 0.0 
Ax 5.0 i S00 S02) Amax = 0-1,4=0.1 
t=1 B N B N B 
x(1) 3.0 DS 0.5 0.0 2.5 f(x) = 42.25 
Vf) =--9.0 —--21.0 0.0 0.0 0.0 
r 0.0 -21.0 0.0 9.0 0.0 
AX 0.0 MO A 0.0 21.0 — Arnax = 0.0238, A = 0.0238 
t=2 B B N N B 
x?) 3.0 3.0 0.0 0.0 2.0 f(x) = 35.00 
Vix)  -10.0  -20.0 0.0 0.0 0.0 
r 0.0 0.0 20.0 -10.0 0.0 
Ax -10.0 10.0 OO E1010, | 100 A max = 0-2, A = 0.125 
t=3 B B N N B 
x) 1.75 4.25 0.0 1.25 0.75 f(x®)) = 28.75 
Vé(x) 15.0 -15.0 0.0 0.0 0.0 
r 0.0 0.0 15.0 0.0 0.0 
AX 0.0 0.0 0.0 0.0 0.0 Stop 


Major and Minor Iterations in Reduced Gradient 


At any point in a reduced gradient search, the superbasic variables represent a “free” 
set in that they can increase or decrease without losing feasibility. A refinement that 
has proved useful exploits this relative ease of movement by dividing the search into 
major iterations and minor iterations. 


A minor iteration leaves nonbasics at bound zero fixed, changing only superbasics. 
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When progress slows, we undertake a major iteration changing more nonbasics as in 
Algorithm 14C. 


Second-Order Extensions of Reduced Gradient 


can be productively extended even further 


Major/minor direction procedure | 
by employing second-order information on the objective function. Thinking of ob- 
jective function f(x) as a function of the superbasics alone, with other nonbasics fixed 
= 0, and basics taking implied values, we are left with an unconstrained optimization 
in the superbasics. Quasi-Newton methods of Section 13.7 can then be employed 
to quickly find a good choice of superbasic values. Afterward, having completed 
several minor iterations, we consider making other nonbasics positive. 


Generalized Reduced Gradient Procedures for Nonlinear 
Constrants 


To this point we have assumed that all constraints of the given nonlinear program 
are linear. Generalized reduced gradient algorithms extend to nonlinear constraints. 
Suppose that we are given the nonlinear equality constrained standard form 


min or max f(x) 
s.t. gi(x) = D; ie€E (14.50) 


Using first-order Taylor approximations, it is natural to consider linearizing con- 
straints around a current x® as 


bi = gi(x) © gi(x) + Vgi(x): (x — x) (14.51) 
Noting that feasibility implies g;(x) = b;, this linearization simplifies to 
Vgi(x) x = Vgi(x)-x® forallie E (14.52) 


Including nonnegativity constraints with system (14.52) yields linear- 
constrained format 


Ax = b® 


x>0 


with rows of A® being Vg;(x) and components of b® equaling Vg,(x)-x®. We 
are now in a position to employ linear-constrained reduced gradient Algorithm 14C 
(or its second-order extensions). 

Dealing with successive systems (14.52) is essentially the strategy of generalized 
reduced gradient algorithms. Still, there is a difficulty. Approximation (14.51) is not 
exact for nonlinear constraints. Thus enforcement of (14.52) is not guaranteed to 
keep x feasible. 

Generalized reduced gradient algorithms address this difficulty by following 
each reduced gradient move with corrector steps to restore feasibility. In essence. 
a penalty function is introduced (see Section 14.5) and a new move is chosen to 
minimized the penalized objective function. Once feasibility is restored, a new move 
can be computed using equations (14.52). 
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*7 QUADRATIC PROGRAMMING METHODS 
A constrained nonlinear program is a quadratic program or QP if its objective func- 


tion is quadratic and all its constraints are linear (definition | 14.9|). In this section 
we investigate special methods adapted to this class of NLPs. 


General Symmetric Form of Quadratic Programs 


It will be useful to express quadratic programs in general symmetric form. 


Notice that nonnegativity and other variable-type restrictions are treated as main 
constraints. 

The assumption that Q is symmetric (= Q") merely simplifies notation. There 
is no loss of generality because a model with asymmetric Q has the same objective 
value as one with symmetric 


Q=5(0+0Q") 


Quadratic Program Form of the Filter Tuning Example 


We illustrate quadratic programming methods with the tiny distortion tuning exam- 
ple model (14.37) (Section 14.6). In vector format| 14.50], the model is 


min 125-+ (-10,-20-x+x(_} r. )s 


st. (1,0)-x>0 
(0,1)-x > 0 (14.53) 
(1,1)-x <6 
(1,0)-x <3 
(0,1)-x <5 


with G = {1,2}, L = (3,4, 5}, and E =9@. 


max f(w) 450—(w,)* + 6w; — (w2)? + 6w2 + 3 
s.t. +wy —-wW. %4+3w3 = Il 

+3w, +2w2 +2w4 = 6 

W1, W2, W3, W4 > O 


Identify elements co, c, Q, G, L, E,a®, and b; of general form | 14.50 |. 
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Analysis: Arranging objective function elements in matrix form | 14.50 | yields cy = 
50, 


6 perindpsgaody ig 
6 Oetie “bgt sare, 
FE gy het Cae ake produniwenn 
0 G2 io. I92 a5 


With E = {1,2}, G = {3, 4,5, 6}, and L = Q, corresponding constraint coefficients 
are 


a) — Ge =Jl, ay 0), by = 


a? = G20.2). bz = 
a® = (1,0, 0,0), b3 =0 
a® — (0,1,0,0), by =0 
a® = (0,0, 1,0), bs = 
a® = (0,0, 0, 1), be =0 


Equality-Constrained Quadratic Programs and KKT Conditions 
It is instructive to begin our investigation of quadratic programming with the pure 
equality case: 

maxormin f(x) Aex+xQx 


14.54 
s.t. Ax =b ( ) 


Here G = L = Gin general form| 14.50], and coefficient vectors a® for equalities 
i € E have been collected as rows of a matrix A. 
With all constraints equalities, Karush-Kuhn—Tucker conditions (principle 


14.21 |) for model (14.54) require no sign restrictions or complementary slackness 
constraints. Furthermore, the objective function gradient (Q symmetric) is 
Vf(x) =¢+2Qx 


and constraint gradients Vg;(x) are the rows of A. Thus KKT conditions for model 
(14.54) reduce to 


Sy, ay; = ¢ + 2Qx 


L 


Ax=b 


What makes pure-equality quadratic programs special is that these conditions can 
be rearranged into a square system of linear equations. 
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IONS FOR EQUALITY QPs 


FOU Karsh ‘uhe Iieker Spidey eonditions for the equality-constrained quad- 
ratic program 


min 4(y1)? — 6y1y2 +5(2)? +3 
st. +y: —3y2 —9y3 = 11 
Vi aye a 1y3 = = 9 


at y = (2,0, —1). 
Analysis: Here 


0 
(134) (8) a o(- 


Thus KKT conditions | 14.51 | are 


—8 GineGicn Tee 


Direct Solution of KKT Conditions for Quadratic Programs 


The unusually simple form of KKT conditions for equality-constrained quad- 
ratic programs suggests an approach to solution. We could simply form the KKT 
system of linear equations and solve for KKT point x and corresponding Lagrange 
multipliers v. 

This is the approach taken in many methods. 


Sophisticated methods of linear algebra may be used to compute answers, but the 
process remains essentially one solving the KKT system. 


Unique solvability of system | 14.51] would mean that model (14.54) has a 
unique KKT point. Since equality constraints assure that every local optimum is 
a KKT point (principle | 14.27|), a unique | 14.51] solution must correspond to a 
unique local (and thus global) maximum or minimum unless the model has no ex- 
trema at all. Other cases may have multiple KKT points, or none at all. Still, any 


local optimum must be a solution to system 


KT CONDITIONS FOR EQUALITY OPs 


Solve the KKT conditions of the Denali? -constrained quadratic program in Sample 
Exercise 14.24 to find a KKT point of the model. 


SAMPLE EXERCISE 14, 25: SOLVI ; 
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Analysis: The unique solution to this KKT system has primal solution 
y1 = —0.0834, y2 = —0.0992, y3 = —1.1984 
and corresponding Lagrange multipliers 
v1 = —0.2486, v2 = —0.1768 


Further analysis would be required to determine whether this KKT solution is a 
global maximum, a global minimum, or a saddle point. 


Active Set Strategies for Quadratic Programming 
Active set methods exploit the linear equation form of KKT conditions for equality- 


constrained QPs by reducing general quadratic programs 14.50] to a sequence of 
equality cases. To see how, define 


S 4 set of indices of active constraints at current feasible solution 
x in general QP model | 14.50 
As 4 matrix with rows formed by the coefficient vectors 
a® ofie S 
Every equality constraint of E belongs to S, along with active inequalities of G and L. 
Suppose that we require all active constraints i € S to continue being satisfied 
as equalities during our next move. Then an optimal move Ax from x” should solve 
max or min f(x® +Ax) = fx®) + Vf (x) -Ax+AxQAx 
s.t. A, AX => 0 


The (14.55) objective merely rewrites f(x +A x) in terms of the second-order Taylor 


(14.55) 


representation (principle| 13.17] [13.17 17 |), which is exact for quadratic functions. Constraints 


enforce the familiar requirements > a;; Ax, = 0 (principle [3.25]) for a move to 
preserve linear equality constraints. 

Notice that subproblem (14.55) is now in equality-constrained format (14.54). 
Thus we can compute a move A x by solving the corresponding KKT linear equations 


14.51]. 


1453 e ‘Active set methods for general quadratic “programs: compute the move 
AX a current solution x” by solvin -arush-Kuhn—Tucker conditions _ 


As with models having only equality constraints, a KKT solution to system | 14.53 
may not exist, or not correspond to the desired minimum for a minimize problem 
or maximum for a maximize problem over the active constraints. Still, construction 


14.53 provides good results when the objective function is reasonably well behaved. 
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To illustrate, return to tuning model (14.53), which is a convex program. At 
x = (2.5, 0), only nonnegativity constraint i = 2 is active, so 


S={2} and As=(0,1) 
With Vf(x®) = (—5, —25), the corresponding move-finding KKT system | 14.53 | is 


—2 A) Ax —5 
2D Ax2 —25 
ORIG V2 0 


This system has unique solution 
Ax = 25); Ax2 => 0, yW2= —30 (14.56) 


Step Size with Active Set Methods 
If a move Ax ¥ 0 results from subproblem (14.55), the usual update 
xD — xO + Ax 


optimizes the objective over the active constraints. However, we have ignored inac- 
tive constraints in forming (14.55). A full step in direction Ax may cause some such 
constraint to be violated. 

To account for this possibility, we introduce a now-familiar maximum step rule: 


omputed from the active constraints at solution x® is nonzero, 
rithms adopt step A in direction Ax, where 


Opec 
ax b : 
>: a Ax <0, ie a| 


—a®Ax 


: b; — ax : 


ke min | 


The first two possibilities for 4 check inactive > and < constraints, respectively, 
and the 1 in the last step provides for the possibility that the full move is feasible. 
For example, we would compute the appropriate step in tuning example direction 
Ax = (2.5, 0) of (14.56) from x = (2.5, 0) as 


AG = +00 


At = min {52, $2} =0.2 (14.57) 


A = min{1, +00, 0.2} = 0.2 


S) aZE IN SET QP a 
Suppose that an active set search of a quadratic program with constraints 
2yi + 3y2 = 10 
ite a0 
1yi +3y2 = 17 


has reached y = (2, 5) and computed (construction | {4.53 |) move Ay = (—3, 1). 
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(a) Determine the appropriate step size i to apply. 
(b) How would the 4 change if the second constraint were ly; + 7y2 < 80? 


Analysis: We apply rule | 
(a) Only the last, equality constraint is active. Changes in the other constraints per 
unit step in direction Ay are 

a .Ay = (2,3)-(—3,1) = -3 

a® Ay = (1,7)-(-3,D = 4 


Thus 
19 — 10 40 — 37 : 
dimitirgrabsd ear grT a = min {1,3, 3} = 3 
(b) With this revised right-hand side, 
59 BO = Sul. 


Ne Sr = , A=min{1,3,f}=1 


4 
Although step sizes up to A = min {3, 3} are feasible, the optimal one of KKT 


computation | 14.53 | occurs at 4 = 1. 


Stopping at a KKT Point with Active Set Methods 

Update x") <— x® + ) Ax advances us toward an optimum to (a reasonable well- 
behaved) general quadratic program as long as move direction Ax # 0. 
Should we stop when Ax = 0? It depends on whether Lagrange multipliers y 


computed from linear system | 14.53 | for active constraints complete a KKT solution 
for the full model. 


the full model | 14.50 because the corresponding optimality conditions are 


C+ 20xPi= Ye ra vp Atv (14.58) 
i 
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and feasibility in primal constraints. But (14.58) is the first part of the linear sys- 


tem solved in | 14.53); complementary slackness is automatic because only active 
constraints are allowed to have v; 4 0; primal feasibility is enforced by the second 


part of equation system | 14.53 | and step size rule | 14.54). Thus the only additional 


requirements for a KKT point are the sign restrictions of principle | 14.55 |. 


constraints 


wi +2w2 >4 
w3 — 8w4+ws <2 
3w, + 2w2 + 2w3 + 2ws + 2ws = 16 
Determine whether the procedure would stop if solution of linear equation system 


produces: 


(a) Aw= (0,0,0,0,0), v= (—33, 10, 14) 
(b) Aw = (0,0,0,0,0), v= (33, 10, 14) 
(c) Aw =(2,-1,-1,0,1), v= (33, 10, 14) 


Analysis: We apply principle | 14.55 |. 


(a) For this maximize model, v; = —33 is appropriate for a > constraint, and v2 = 10 
is suitable for a <. The search would terminate with the current solution a KKT 
point. 


(b) For this maximize model, vj = 33 violates sign restrictions of| 14.55). The search 
would not terminate. 


(c) Here the move direction Aw + 0. The search would continue. 


Dropping a Constraint from the Active Set 


Clearly, the active set must change if further progress is to be achieved when sign 


restriction | 14,55 | are not fulfilled even though system | 14.53] produced a move 


Ax = 0. In particular, one or more now active constraint i €¢ S must be allowed to 
become a strict inequality. 
To see which active constraint to drop from S, focus again on the Lagrange 


multipliers computed in solving system | 14.53]. We know these multipliers can 
be interpreted (principle | 14.17|) as the change in (14.55) optimal value with its 
constraint right-hand sides. For example, v; > 0 for a < inequality i of a minimize 
subproblem indicates that allowing inequality i to become strict (i.e., relaxing to 
a. Ax < 0) will help the objective function. That is, a violation of sign conditions 


14.55 | indicates an active constraint that could be productively dropped. 
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14.28: DROPPING CONSTRAINTS IN ACTIVE 


SAMPLE EXERctS Ser QP 


For each of the cases in Sample Exercise 14.27 where the procedure did not terminate, 
determine how the active set § should be modified. 


Analysis: We apply principle 14.56, ; 


(a) The procedure stops in this case and no modification of S is required. 


(b) For this case we drop constraint i = 1 from S and re-solve linear system 1453 
because Lagrange multiplier vy; = —33 violates sign restrictions | 14.55 |. 


(c) No change is needed in S for this case because the move direction A w + 0. 


Active Set Solution of the Filter Tuning Example 


Algorithm 14D collects principles | 14.53 | to 


quadratic programs. Figure 14.10 and Table 14.6 detail the application of Algorithm 
14D to tuning model (14.53), beginning at x = (2.5, 0.0). 


ALGORITHM 14D: ACTIVE SET METHOD FOR QUADRATIC PROGRAMS — 


Step 0: Initialization. Pick starting feasible solution x, and initialize working 
active set S with indices of all constraints active atx. Also, choose stopping 
tolerance €>0, and initialize iteration index t <0. 

Step 1: Subproblem. With As 4 coefficient matrix of active constraints in 
S, solve the Karush-Kuhn-Tucker conditions 


-2Q A! Ax \ ff Vf) 
As 0 yar G 0 


of direction problem (14.55) for move direction Ax‘*! and active constraint 


Lagrange multipliers vy*”. Lagrange multipliers for nonactive i ¢ S are 
fixed vet? <0. 
Step 2: KKT Point. If ||Ax“t)|| < ¢ and v“® satsifies sign restrictions of 


1455 
symmetric quadratic program | 14.50]. Otherwise, if ||Ax“+ || < e, go to 
Step 3, and if not, proceed to Step 4. 

Step 3: Active Dropping. Choose some i € S with Lagrange multiplier 


, Stop; the current x® is a Karush-Kuhn—Tucker point of the given 


v*) violating sign restrictions , and remove it from S. Then go to 
Step 6. 

Step 4: Step Size. Compute the maximum appropriate step 4 in direction 
Ax™? via 


14.7 Quadratic Programming Methods 857 
OO 
CRED. Cli b; F 
Tia t+1 ; 

AG < min | goat ralAx’™ <0, i¢€ c| 
b; —a®x® 
min ; ————— 
| a®@ Ax 


Xr < min{l,rAG, Az} 


pee -a Ax! 5 0, ve 1 


Step 5: Move. Step 


xD 2 yO 1 AyD 


and update S$ with the indices of any newly active constraints. 
Step 6: Advance. Increment ¢ —¢+1, and return to Step 1. 


FiGuRE 14.10 Active Set OP Solution of Filter Tuning 
Example 


TABLE 14.6 Active Set OP Solution of Filter Tuning Example 


Variables Constraints 
x X #21 G=2 123, d=4 t=5 


t=0 x9 25 0.0 
Ax 2.5 0.0 
Podge xe 30 0.0 
Ax 0.0 0.0 
t=2 x®@ 3.00 0.00 
Ax 0.0 13.0 


no yes no no no 
0.0 -30.0 0.0 0.0 0.0  A=0,2000 
no yes no yes no 
0.0 -26.0 0.0 40 00 dropi=2 
no no no yes no 
0.0 0.0 0.0  -300 00 A=0.2308 


t=3  x@) 3.00 3.00 no yes yes no 

Ax 0.0 0.0 0.0 0.0 -20.0 10.0 0.0 dropi=4 
t=4 x4 3.00 3.00 no no yes no no 

Axe 25 ai 25 0.0 0.0 —15.0 0.0 0.0 A=1.0000 
t=5 x) ITS 4.25 no no yes no no 


4 HAl/4¢ Ale Ale Ale Ale WH 
=] 
° 


Ax 0.00 0.00 0.0 0.0 -15.0 0.0 0.0 KKT point 
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Computations (14.56) and (14.57) have already established the first move di- 
rection Ax = (2.5, 0.0) and step size 4 = 0.2, which lead to 


x =x 4 Ax = (2.5, 0.0) + 0.2(2.5, 0.0) = (3.0, 0.0) 
The active set S = {2, 4} at x includes the previous x. > 0 and newly active 


x1 < 3. Solution of the corresponding linear system | 14.53 | produces null direction 
Ax = 0. We have not yet reached a KKT point for the full model because the sign 


of v2 = —26 is wrong for a > constraint in a minimize problem (principle | 14.55]). 
Thus we drop i = 2 from S and re-solve equations | 14.53 |. 


The new direction produces a nonzero move, and the search continues. Att = 5 
the computed direction is again Ax = 0. This time, however, Lagrange multipliers 


satisfy sign restrictions of principle | 14.55]. The search terminates with KKT point 
(here global optimum) x* = (1.75, 4.25). 
14.8 SEPARABLE PROGRAMMING METHODS 


Separable functions decompose into sums of functions of single decision variables 
(definition ), and separable programs are NLPs over separable objective func- 
tions and constraints (definition ). That is, they take the general form 


max ormin f(x) 4 = nes) 


j 
s.t. gi(x) & Ne Bij) = Dd; forallieG 
vi 


IA 


si 2 S> giej) < bi  forallieL (14.59) 
j 

gece S- sij Oe) 
j 


xj = 0 for all 7 


bd; for allic E 


Here G, L, and E index the >, <, and = constraints, respectively. For notational 
convenience, we also assume that nonnegativity constraints apply to all variables. 


Pfizer Example Revisited 


Section 14.2 developed Pfizer pharmaceutical manufacturing lot size model 
; 2160 8712 
min 66.21x; + —— + 426.8x, + —— (total cost) 
x1 x2 


330 2916 
61.20 — + 268.1 — 
bj aa x3 as va xen (14.60) 


S.t. 


180° 726 227-582 243 
+ + + SPINS) (production time) 
XY x2 X3 X4 


Nip een 10) 
Variables in this example are 


x; 4 number of batches in each run or lot of product j 


14.8 Separable Programming Methods 


It is easy to see that this model is separable because its objective function can 
be written 


F(%1, X2, x3, x4) A fi(x1) + fare) + fa(%3) + fara) 


where 


2160 

fir) A 66.21%, + — 
x1 

8712 


x2 


fol) 2 426.8x9 + 


330 
fa(%2) A 61.20x3 + — 
x3 


ih 
falx2) & 268.1%4 + — 
4 


and the main production time constraint decomposes similarly with 


gi%1) = = 
82(%2) = = 
§3(x3) 2 = 
g4(x4) & = 


The remaining four (nonnegativity) constraints are linear and thus automati- 
cally separable. 


By definition, linear functions consist of a sum of terms a;x; involving single decision 
variables. 


Derr whether each of the following functions is separable. 
(a) f(w1, w2) & (wy)3° + In(w2) 
(b) g1(w1, W2) 2 14; — 26w2 


(c) go(wi, W2) A 14m, + wiw2 — 26w2 


Analysis: We apply definition 


(a) This f is separable because it decomposes into the sum of f,(w1) 4 (w,)°° and 
fo(w2) & In(w2). 
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(b) This g; is separable because it is linear (principle | 14.57 ]). 


(c) This go is not separable because the term w;w> involves both variables. 


Piecewise Linear Approximation to Separable Functions 


The main special convenience of separable programs is that they can sometimes 


be approximated closely by LPs (principle | 1 
piecewise linear approximation of the 1-variable functions fj and gi. 


|). The transformation begins with 


14.58 | Piecewise li 
main of each decision variable x 
line, 


: 


Figure 14.11 illustrates for x; in our Pfizer model (14.60) over domain [0, 8]. 
Both objective function term f,(x;) and main constraint function g1(%1) have been 
approximated with three linear segments. One covers x; € [0, 1,1] = [0, 1];a second 
treats x1 € [u,1, 1,2] = [1,5.7]; and the third handles x, ¢ [u41,2, 41,3] = [5.7, 8]. 
First breakpoint u;,, = 1 was chosen near the “knee” of the two functions, uj. = 
5.7 approximates the minimum of f,(x)), and u;,3 = 8 is a practical upper bound 
on Xj}. 


fils) = 66.21x, + 2260 


1 
180 
810%) = x 


Sis = 23107 
BSG 


ay, = -31.6 


U4, t= 1 uy42 = uy, a 8.0 


FiGureE 14.11 Piecewise Linear Approximation of Pfizer Example 
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Introducing new variables x1,1, 1,2, and x13 with coefficients as in Figure 14.11, 
we have 


2160 
fi (x1) A 66.21% + — & 6480 — 4254x141 ze 312.7x1,2 + 18.8x1,3 
x1 


180 


E: x 1540 — 1360x1,1 = 31.6x1.2 = 3.9x1.3 
1 


gi(%1) 


Notice that the same interval limits must be used to approximate x, in the 
objective and all constraints. Upper bounds on the new variables are derived from 
interval limits 


0<x%11<m1=1 
0< X12 £12 -—W41 = 5.7-1.0 = 4.7 
O< 13 £413 —-W4,2 = 8.0 -—5.7 =2.3 


say si 


Gonster a eon plinear program with abigeive Aingtion and pone com- 
ponent functions for nonnegative decision variable w; given by 


fii) & (wi)? — 4m, +22 
gii(wi) 2 Jw + 9 
g2,1(W1) 2 1494 


Form corresponding piecewise linear approximations using breakpoints u1,,; = 2 and 
u42= 5. 


tercepts are 
¢10 = fi) = 22, ai10 = 8110) =3, 42,1,0 = 82,100) =9 
Slopes for the interval [0, w1,1] = [0, 2] are derived as 


_ fi2)- fi) 


C11 => oe 0 = —2 
£1,1(2) — 81,100) _ 
a4 = ae = 0.158 
82,1(2) — 82,100) 
- 14 
42,1,1 2-0 


Corresponding slopes for interval [u1,1, u2,1] = [2, 5] are 


fi) =e) 


= = 5 
a 5=2 
81,16) — 8112) _ 
411.2 = a) = 0.142 
5) -— Dp) 
ee 82,105) — 2,1(2) ei, 


Sa 
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Thus piecewise linear approximations are 
fii) © 22—2wi1+3w12 
81101) © 3 +.0.158w11 + 0.142012 
§2,1(W1) © 04+ 1411 + 14012 


Linear Program Representation of Separable Programs 


Applying piecewise linear approximation | 14.58 | to every variable of separable pro- 


gram format (14.59) yields an LP approximation. 


This model is a linear program solvable by methods of Chapters 5 and 6. 


Full application of construction | 14. 
the linear approximation 


min 42,354 — 4254x)) — 312.7x1,2 + 18.8x1,3 
— 16997x2,1 — 1509x2,9 + 184.8x23 
— 598.8x3.1 — 82.3x3.2 + 43.3x3,3 
— 5564x4,1 — 615.5x4.2 + 157.6x43 
st. 7030 — 1360x1,; — 31.6x1,2 — 3.9x1,3 
— 2452x921 — 161.3x22 — 20.2x23 
— 555x3,1 — 12x32 — 1.5x3,3 
— 1486x41 — 73.6x4,2 — 9.2x4.3 < 221.5 
(earl, Vena, 0 x13 <2.3 
O<%x21 <1, 0O< x22 < 3.5, 0< x93 <3.5 
O< X31 5 1, 0 <%325 3, 0 < X33 5 ei 


O27 = 1,0 2s 223, ee a7 


) | to our Pfizer lot sizing model produces 


(14.61) 


de Te Ot 8 eR ee i ek ee ee 
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An optimal solution is 
Xt 4 = OF XT 9 = 47, XT 3 =12.9.05 OF Xt 4 +Xx75 +73 = xj} = 8.0 
%,=10, 1,535, x3,=2.4, or x3, +43, +45,=%5 = 6.9 (14.62) 
*3, =1.0, x3,=1.3, x3,=0.0, or x3,+23,+2x3,=x3 =23 
x41 = 1.0, x4, =2.3, x4,=0.0, or x4, +24, +443 =19 =3.3 


which corresponds fairly well to the nonlinear optimum of (14.7). 


Correctness of the LP Approximation to Separable Programs 


Does linear program] 14.59 |correctly model (14.59) (except for interpolation error)? 
Sometimes yes, sometimes no. To see the potential difficulty, suppose that we attempt 
to 

max f(y) Sy? —12y+45 

s.t. O<y<9 
This f is the function in Figure 14.4 (Section 14.2). Obviously, the optimal y* = 0, 
because f(0) = 45, f(9) = 18, and every y in between has a lower function value. 

Forming linear program representation | 14.59 | for this simple example yields 


max 45 —7y,+2y2 
Ste OES tyre 
Oy res 
An optimal solution makes y; = 0, y3 = 4, which implies that y* = yi + y3 = 4. 
What went wrong? The approximation 


fO) co+ D> cure 
k 


corresponding to} 14.58 | is correct only if we assume that segment variables y; satisfy 
a certain sequence at optimality. For y € [0, 5] we want first segment y, to represent 
y. If y € [5, 9], we want segment 1 to run to its upper limit and segment 2 to do the 
rest. 

In general, each segment of a piecewise linear approximation must reach its 
upper bound before the next is available for use. 


Pfizer example results (14.62) illustrate a case where conditions | 14.60/ are 
satisfied. For example, at 7 = 2, segment upper bounds in formulation (14.61) are 
1.0, 3.5, and 3.5, while 


X34 = 1.0, X39 =s)55 X33 =2.4 


Each positive segment has the preceding one at its upper bound. 
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A piecewise OFnear approximation to a separable nonlinear program in 1 nonnegative 
variables w; and w2 uses breakpoints u;,; = 2 and u;,. = 6 for the first variable, 
together with uz, = 7 and uz. = 20 for the second. Determine whether each 
of the following LP approximation solutions provides a correct answer (except for 
interpolation error) to the original nonlinear program. 


aero — haere it Fe 
(a) Wii = Ds Wi = 3; W314 = 6, Wo = 0 


(b) wi, =0, Win = Bk W1= 1,w3,=13 


Analysis: Following formulation A .59|, bounds on the segment variables will be 


0<w1 <2-0=2, 0<wi2=6-2 = 2) 
0< Wi 37-027, sO = 20-76 


(a) This optimal solution does give a correct approximation because it satisfies se- 
quencing conditions | 14.60}. Variable w;, equals its upper bound, so w;, can be 
positive. 

(b) This solution yields an incorrect approximation because it violates sequencing 
conditions [14.60]. Variable w7, > 0 and w7 , does not equal its upper bound. 


Convex Separable Programs 

Suppose now that the given separable program (14.59) is also a convex program. 

That is, we are minimizing a convex objective or maximizing a concave one, subject 

to concave > constraints, convex < ones, and linear equalities (definition | 14.3 | ). 
These requirements relate to the entire objective of constraint functions. Still, 

it is easy to see that all component functions for each variable in separable form 

(14.59) must have similar properties. 


14. 61 Separable function 


We already know from principle | 13.29 | that sums of convex (or concave) functions 


are convex (concave). But| 14.61 | asserts that the converse is also true for separable 
functions. To see why, we need only choose points x and x with all components 
equal except the jth. Then a step 4 from x” toward x® changes just component 
Sj Cs + Ma — a) of s. Convex function definition | 13.23 
holds for s;. 


|can hold for s only if it 


14.8 Separable Programming Methods 


Given their many convenient properties, it should not surprise us that separa- 


ble convex programs satisfy requirement 14. 
programming. 


60 | for good approximation by linear 


14.62 | 14.62, ae Linear approxi allo : $ 


= imatic separable convex programs have an 
iditions | 14.60 if they have any opti- 


Mum at all. 


optimal solution satisfying sequenci 


Again, Pfizer example model (14.61) illustrates. We have seen in Section 14.2 
that the nonlinear version (14.60) is a convex program. Thus it was no accident that 


segment optima (14.62) satisfy sequencing condition | 14.60 |. 
To understand why separable convex programs have the required property for 


effective LP approximation, recall (principle [13.27]) that second derivatives of con- 
vex functions are nonnegative and those of concave functions are nonpositive. It 
follows that first derivatives, or slopes, are nondecreasing and nonincreasing, respec- 
tively. 

For a minimizing convex objective, this implies that the least cost slope Chk 
of each approximation occurs at k = 1, and for a maximizing concave objective, 
the first segment will also be the most preferred. In a similar way, coefficients in 
constraint rows also exhibit a preference for lower-numbered segments. For example, 
if constraint i is <, and thus convex, aj: is the smallest slope in the approximation of 
8ij- Thus it does least damage to feasibility. If the constraint is >, and thus concave, 
dij, 18 largest and advances feasibility most rapidly. 

Combining these observations about objective and constraint approximations, 
we see that the first segment x;, of each piecewise linear approximation gives the 
greatest objective function payoff with the least burden on constraints. An optimal 
LP solution must choose it first, making the second segment positive only when the 
first has reached its upper bound. The pattern continues through each interval k, so 


that property | 14.60 | is satisfied. 


Difficulties with Nonconvex Separable Programs 


When the given separable program is not convex, property | 14.60) may not hold 
automatically. However, it can be enforced artificially. 
Suppose that we apply upper and lower bounded simplex Algorithm 5E to lin- 


ear approximation | 14.59 |. Each iteration chooses either a nonbasic lower-bounded 
variable to increase or a nonbasic upper-bounded variable to decrease. 

General separable programming searches simply restrict the choices even fur- 
ther to maintain sequencing property | 14.60}. A nonbasic lower-bounded segment 
variable cannot increase unless the preceding segment is already at its upper bound. 
Similarly, a nonbasic upper-bounded segment variable cannot decrease unless the 
succeeding segment has value 0. 

Of course, these extra limitations may prevent the simplex from computing an 


optimal solution to linear program 14.59 . However, they do assure that property 
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is enforced. When no allowable pivots remain, we stop with a heuristic 
optimum. 

An alternative approach producing globally optimal solutions can be derived 
via integer linear programming methods of Section 11.1. Binary variables y,; are 
introduced that parallel each x; x, with 


sale tix, = 0 
Vik = £ 
0 otherwise 


Then the switching constraints 
(uj = Uj,k-1) Vik SX (ujx = Uj et) Vik for all j, k 


enforce sequencing conditions | 14.60 by pushing each segment to its upper bound 
if the next is positive. 


14.9 POSYNOMIAL GEOMETRIC PROGRAMMING METHODS 


As we have seen in Sections 14.1 and 14.2, many important applications of nonlin- 
ear programming arise in engineering design, where decision variables are physical 
dimensions, pressures, and so on. Such models are often highly nonlinear and have 
many locally optimal solutions. This section deals with special cases called posyn- 
omial geometric programs, which address the “variables to powers” form of many 
engineering design models and constitute the only broad class of nonconvex NLPs 
readily solved to global optimality. 


Posynomial Geometric Program Form 


In Section 14.2 we introduced (definition | 14.10 |) posynomial functions, which are 
positive-weighted sums of products of decision variables raised to arbitrary powers. 
A posynomial geometric program (GP) is an NLP minimizing a posynomial objec- 
tive function over positive variables and < posynomial main constraints (definition 


). The general form is 
min > aT] () 


keKy fal 
Ms 14.63 
s.t. sD dy | [| <1 i ee em ( ) 
kéK; fel 
xj>0 pSoos ie 


where nonoverlapping sets K; index the posynomial terms in the objective and con- 
straints, values d; are the corresponding weights, and the a;,; are exponents of vari- 
ables x; in terms k. 

Several details of the format will prove critical to its tractability: 


e The objective must minimize a posynomial. Maximizations are not allowed. 

e Coefficients d, must all be positive. 

e Constraints must enforce < requirements ona posynomial. Any = and > forms destroy 
the structure. 

e Decision variables must be limited to positive values. We will want to take their 
logarithms. 


14.9 Posynomial Geometric Programming Methods 


Cofferdam Example Revisited 


We illustrate GP methods with Section 14.2’s model (14.11) to optimize a cofferdam 
design: 


XX: 1.46 x 107 
1 2 


min 168x;x2 + 3648x, + 3648 = (cost) 
3 X4 

1.25. 41.625 

st. 5 el (empirical) 
a i (14.64) 
1.0425. 
ee eal (slipping) 
x2 
0.00035x1.x3 reall (tension) 


X1,X2,%3,X4 > 0 


In the notation of (14.63), Ko = {1.2.3.4}. Ki = {5,6}, Ko = {7}, and K3 = {8}. 
Corresponding coefficients are 

d, =168, d, = 3648, d3 = 3648, dg = 146 x 10' 

ds =1.25, do = 41.625, d;=1.0425, dg = .00035 


ai= 1, mo2= 1, a3= 0, ag= O 
i= 1, m2= 0 m3= 0, ms= 0 
@ai= 1, @.= 1, @3=-1, a@,= 0 
a4, = 0, 442 = 0, 443 = 0 14,4 = —1 
ai1=—1, a52= 0, @3= 0,. G@4a= 1 
46,1 = —1, 462 = 0 46.3 = 0, 164 = 0 
a7,) = ile! 42 = —1, a73= 0 a7,4 = 0 
ag,= 1, ag2= 0, ag3= 1, agg= 0 


_ SAMPLE EXERCISE 14.32: PLACING GEOMETRIC) OR 
ee Form 


Identify the constants and index sets of standard form (14.63) for the following posyn- 
omial geometric program: 


ws 
: 1 
min 3— + 14w2w3 
Ww2 


st. W1./W3 + W2./W3 < 20 


Wi 


<1 


W2 
W1,W2,W3 > 0 


Analysis: Begin by dividing through the main constraint by 20 to obtain standard 
right-hand side 1. Then the objective function has terms k € Ko A {1, 2}, the first 
constraint involves k ¢ K; A {3, 4}, and the second has only the one k € K> A {5}. 
The corresponding standard-form coefficients are 
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d,=3, dg=14, d3=0.05, dg=0.05, ds =1 
and 


ai= 0.43, yA2= —1, 43 = 0 


a1 =0, 2= 1, 3= 
a1 = il; a2 = 0, 43.3 = 5 
41 = 0, 42 = 1, a43 = 55) 
a5, =1, a452=—1, a53=0 


Logarithmic Change of Variables in GPs 


Posynomial functions need not be convex [see example (14.14)], and thus geometric 
programs (14.63) are often not convex programs. However, a change of variables 


), we consider substituting 


(14.65) 
or 
xj 3 (14.66) 
Under transformation (14.66), terms k of posynomials in (14.63) simplify as 
dx I] Cen = dk I] (2) = dx ssi etki = deri i = der” “2 
where a 4 (ay1,...,a¢n) andz A (z1,...,z,). Then original geometric program 
form (14.63) becomes 
iin | 7) 2" eee 
keKo 
st.  g;(z) A a den” <1 i=1,...,m (14.67) 
keK; 
zj URS jaiee aun 


For example, cofferdam model (14.64) transforms to 


min 168e*” ** + 3648¢8” *7 + 3648e8° * 7 + (1.46 x 107)e8® 2 
gt, ase * aig os a eae 


1.04258 *2 = 4 (14.68) 
0.00035e8" * 2 <1 
z URS 


with 
a) = (1,1,0,0), a®=(1,0,0,0) 
a® = (1,1,-1,0), a” = (0,0,0, —-1) 
a® = (-1,0,0,1), a® = (-1,0,0,0) 
a? — (1,-1,0,0), a® = (1,0,1,0) 
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Return to the posynomial geometric program of Sample Exercise 1432. Change 
variables via (14.66) to produce a convex program in format (14.67). 


Analysis: Using the coefficients of Sample Exercise 14.32, the transformed model is 


min 30-4321 —lza af: 14eletles 
St. _ 0.05¢e'%1 +0523 4.0,05¢124052s < 1 


Jelu—lza < 1 


Z1, Z2, 23 URS 


Convex Transformed GP Model 
The power of this simple change of variables (14.66) is to convert a posynomial 
geometric program to a convex program (definition | 14.3 }). 


“tating: = = a is a conve program in new variables z;. 


To see why the transformed model is convex in z, observe that the objective and 
constraint functions are positive-weighted sums of terms 


aez 


D(z) Ze 
Linear exponent a“ - zis convex inz (principle| 13.28 |), and h(y) A e” isnondecreas- 


ing and convex. Composition principle then implies that each p; is convex, so 
that their transformed sum must be, too. It follows that format (14.67) minimizes a 
convex objective, subject to convex < constraints, which makes it a convex program. 

Notice how this analysis depends on details of the posynomial GP format. 
Coefficients d; , must be positive for weighted sums of convex terms to be guaranteed 
convex. Also, a minimizing objective and < nonlinear constraints are essential if a 
model over convex functions is to be a convex program. 


Direct Solution of the Transformed Primal GP 


Principle | 14.63 | provides a direct avenue to computing global optimal solutions to 
posynomial geometric programs (14.63). We need only substitute x; = e*/, solve the 
resulting convex program in z by methods of Section 14.6, and transform back by 
xi <e% for allj (14.69) 
For example, application of reduced gradient Algorithm 14C to transformed model 
(14.68) produces optimal solution 
= 4.188, z =4179, z= 3.820, 24 = 2.823 

Then inverse transformation (14.69) gives the following optimal solution in original 
variables: 

foe = Ops, ae = O.32. 

goa 60, ae = 16.82 
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Dual of a Geometric Program 


Sometimes even more efficient methods than solving convex program (14.67) can be 
used to optimize geometric programs. The process begins with still another transfor- 
mation of the given model termed its dual. 

In addition to the usual Lagrange multiplier for each constraint, geometric pro- 
gramming duals introduce variables for every term k of the objective and constraint 
posynomials: 


v; 4 Lagrange multiplier for constraint i 


6, 4 dual variable for posynomial term k 


168 3648 3648 
max 6,In{ —]+6)In + 63 In 
61 62 53 


1.46 x 107 1.2. 41. 
t sain ( S ) + sin (=) + sim ( = 
64 65 56 


1.0425 : 
+ 5yin( ) am (T=) 
67 dg 


(14.70) 


v1 In(—¥1) — v2 In(—v2) — v3 In(—v3) 
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S.t. +d; +62 £163 051660) id7t = 6g =" 0 
+61 + 53 — 67 = i) 
— 63 gi = 110 
— 64 +45 — a) 
=O) sect Ooi ot 03, 104 eel 
=t=OS tect tOG =-V; 
= 07 ==) 
+63 =—V3 
d1,...,63 > 0 


V1, V2, V3 <0 


It first 4 constraints weight 6, with exponents aj, for primal variables j. The fifth 
normalizes the 6 total of variables relating to the objective function. The remaining 
main constraints simply recover (negatives of) Lagrange multipliers for the 3 primal 
constraints as sums of associated 6,. An optimal solution is 


vi = —1.225, v5 = —0.481, vs = —0.155, 
5* = 0.326, 8 =0.108, 8 =0.155, 6 = 0.411 (14.71) 
s§=0.411, 8*=0814, 5 =0.481, 8% = 0.155 


with optimal value 14.563. 


| SAMPLE EXERCISE 14.34: FORMULATING GEOMETRIC PROGRAM DUALS _ 


Form the dual of the posynomial geometric program in Sample Exercise 14.33. 


the main constraints, and variables 5, ... , 5s for the five posynomial terms. Then the 
dual becomes 


14 f ; il 
max 46;I1n i + 62In {| — ) + 63In O05 + 641n 0.05 + 65 In | — 
oT 62 63 54 55 


vy In(—v1) — v2 In(—v2) 
s.t. 0.436, + 163 + 165 = 0 
—16; + 16, + 164 — 155 = 0 


152 + 583 + .554 = 0 
16; +16, =1 

163 + 164 = -v, 

165 = —v2 

54,...,65 = 0 


V1,v2 <0 
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Degrees of Difficulty and Solving the GP Dual 


Dual problem | 14.64] is a separable program (definition | 14.7|) with linear con- 
straints. Furthermore, its objective function can be shown to be concave over feasible 
(6, v), even though it is not concave over all choices of the decision variables. 


4 04 His a separable ¢ convex Pro- 


Thus either the separable programming methods of Section 14.8 or the reduced 
gradient algorithms of Section 14.6 can be employed to compute a global optimum. 

Sometimes the task is even easier. Noting that the last main system of con- 
straints entirely determines the v; in terms of the 5,, the degree of difficulty of 
a model depends on the number of truly independent variables 6, in other main 
constraints. 


The first two sets of main constraints in dual have one variable for each 
k and (n + 1) constraints for the n primal decision variables. Thus the degree of 
difficulty bounds the number of 5-variables that must be fixed in value to determine 
the rest uniquely. Some models even have degree of difficulty zero, meaning that the 
dual can be optimized by solving a system of linear equations. Cofferdam example 
dual (14.70) has degree of difficulty 


terms — variables —1 = 8 —4—-1=3 


SAMPLE EXERCISE 14.35; DETERMINING GP DEGREES oF Dirricunn 


Determine the degrees of difficulty in poudeanel geometric program of Sait 
Exercises 14.32 to 14.34. 


Analysis: The model has 5 posynomial terms and 3 variables. Thus its degree of 
difficulty is5 -3-—1=1. 


Recovering a Primal GP Solution 


is the model of true interest. Lio can we retrieve a primal sethiam x*? 


An elegant dual relationship makes recovery straightforward when optimal 
Lagrange multipliers are known for the dual. 


14.9 Posynomial Geometric Programming Methods 


For example, optimal Lagrange multipliers for the first 4 constraints of coffer- 
dam model dual (14.70) are 


zt = —4.138, zi = —4.179, zi = 3.820, z¥ = —2.823 


Application of transformation | 14.67 | recovers the same primal optimum that we 
have seen before: 


xt = e438 — 62.65, x3 = e417 = 65.32 
xf = 3820 — 45.60, xj = e283 = 16.82 
Derivation of the GP Dual 
Why is problem | 14.64 


construction 
the objective function of transformed model (14.67). 


min In(f(z)) 4 In (x dew” ) 


keKo 
(14.72) 
st. In(g;(z)) A In (= dees 6 ‘ <0 i=1,...,m 
keK; 
z; URS epic 2am 


Minimizing In(f) is equivalent to minimizing f, and g;(z) > 0, so that logarithms 
always exist. 

Noting the convexity of transformed model (14.67), this logarithmically retrans- 
formed form (14.72) is also a convex program. Karush—Kuhn-Tucker conditions will 


be sufficient for an optimal solution (principle | 14.26 
With Lagrange multipliers, v;, the main rows af KKT conditions are 


s Do ae 7 ae — > dea je" *=0 forallj (14.73) 


keKo 8i @ keK; 


Now substituting 


ie ela (duet “4 k€Ko 


| 
= 
oF 


(14.74) 
bk A (dyet” **) k € K;, i al leben gi) 
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equations (14.73) become 


s ax jo. =O forallj=1,...,n 
al k 


These are exactly the first main constraints of dual | 14.64 |. Sign restrictions on the 


v; and 8, also correspond. 
To make new variables 6, perform according to their definitions (14.74), we 
must also enforce 
f(z) = SS d,e*” “Zz 
keKo 
or, dividing by f(z), 
1= ye ye (14.75) 
keKy 
Similarly, 
i@= ) dye” “Hovalli 24, 
keK; 
becomes upon multiplication by —v;/g;(z) 
—vi = > & for alli=1,...,m (14.76) 


keK; 


Expressions (14.75) and (14.76) complete the constraints of dual | 14.64. 

Only the complementary slackness part of KKT conditions now remain to be 
formulated. Instead of explicitly including such conditions, dual formulation | 14.64 
maximizes objective function 


aio: n(S ) >> viln(—v) (14.77) 


all k i 
over (6, y), fulfilling the other constraints. A long but tedious derivation can show 
that this maximizes the Lagrangian of (14.72) over stationary points z, which has the 
same effect as enforcing complementary slackness. 


Signomial Extension of GPs 


Often, models that cannot be formulated as a posynomial geometric program do fit 
a less restrictive signomial form. 


Note that term weights d, are not required to be positive. 
Obviously, the easy convexity of NLPs over transformed posynomials (princi- 


ple | 14.63) is lost when weights may be negative. Still, considerable tractability is 


preserved. The reader is referred to more advanced books on nonlinear program- 
ming for details. 


EXERCISES 


14-1 A lidless, rectangular box is to be manufactured 
from 30- by 40-inch cardboard stock sheets by cut- 
ting squares from the four corners, folding up ends 
and sides, and joining with heavy tape. The designer 
wishes to choose box dimensions that maximize 
volume. 


(a) [| Formulate this design problem as a con- 
strained NLP. 

(b) [| & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-2 A partially buried, rectangular office building 
is to be constructed with a volume of at least 50,000 
cubic meters. To minimize energy for heating and 
cooling, the exterior roof and sidewall surface ex- 
posed above ground should not exceed 2250 square 
meters. Within these limits, the designer wishes to 
choose dimensions that minimize the volume exca- 
vated for the buried part of the building. 


(a) Formulate this design problem as a constrained 
NLP. 

(b) [9] & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-3 A company maintains inventories of its 5 prod- 
ucts, replenishing the stock of an item whenever it 
reaches zero by manufacturing a fixed lot size of new 
units. The following table shows the setup cost for 
manufacturing, the unit volume, the unit annual in- 
ventory holding cost, and the estimated annual de- 
mand for each item. 


Product 


Setup 300 120 440 190 80 


Volume 33 10 12 15 26 
Holding 87 95 27 36 135 


Demand 800 2000 1350 


Managers wish to choose lot sizes for each item that 
minimize total average annual setup and holding 
costs while assuring that the maximum combined 
stored volume will not exceed the 4000 cubic me- 
ters available. Assume that lots arrive the instant 
they are ordered. 


875 


Exercises 


(a) [| Formulate this operations problem as a con- 
strained NLP. (Hint: What is the average on 
hand inventory of item j as a function of the 
lot size for j?) 

(b) | & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-4 A print shop plans to maintain 5 different 
presses, replacing each every few years on a regular 
cycle. The following table shows the replacement 
cost (in thousands of dollars) of each press, and the 
estimated annual income (in thousands of dollars) 
that each can generate when new. However, as the 
presses grow older, their productivity declines; final 
values in the table show the (simple, not compound) 
percent income loss each year of life. 


Press 
1 2 3 4 5 
Replace 110 450 150 675 320 


Income 90 110 5p) 220 250 
Decline 5% 20% 30% 20% 40% 


The owner wishes to choose a replacement (cycle) 
time for each press that minimizes total replacement 
and lost income costs within the $250,000 she can 
average annually for purchasing new presses. 


(a) Formulate this replacement problem as a con- 
strained NLP. (Hint: What is the average in- 
come loss on press j as a function of the replace- 
ment time for j?) 

(b) [Q] & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-5 A machinist will remove excess metal from a 
rotary (round) machine part by passing the cutting 
tool of a lathe along 42 inches of the part length. 
For a lathe turning at N revolutions per minute and 
advancing the tool at a feed rate of f inches per revo- 
lution, classic empirical relationships project the ef- 
fective life of a cutting tool (in minutes) at 


6.667 
tool life = (age) 


Each time a tool wears out the operator must install a 
new one and spend 0.1 hour realigning the machine. 
Engineers wish to choose the machining plan that 
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minimizes total cost at $52 per hour for machinist 
time and $87 each for new tools. Speed N must be in 
the interval [200, 600] and feed rate f in the interval 
[0.001, 0.005]. 


(a) [| Formulate this machining problem as a con- 
strained NLP. 

(b) [| & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-6 A warehousing firm services orders for its 5 
products from an automatic storage and retrieval 
(ASAR) system, refilling storage from backup ar- 
eas whenever the ASAR stock of any item reaches 
zero. The following table shows the weekly demand 
and the unit volume (cubic feet) for each product. 


Product 
1 2 3 4 5 


Demand 100 25 30 50 200 
Volume 2; 5 3 7 5 


Managers wish to decide how many of each item to 
accommodate within the 1000 cubic feet of available 
storage to minimize the total number of refilling op- 
erations per week. 


(a) Formulate this operations problem as a con- 
strained NLP. 

(b) [6] & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-7 A solid waste company must locate 2 disposal 
sites to service the demand (tons per day) of the 5 
communities detailed in the following table. 


Community 
1 DT Ga eS 
Demand 60 90 35 85 70 


E-W coordinate 0 4 30 20 16 
N-S coordinate O30 S50 se AS 


Each site will be able to handle up to 200 tons per 
day, and planners want to select the site locations to 
minimize to total ton-miles of hauling from commu- 
nity (mile) coordinates shown in the table to disposal 
sites. Assume that hauling distance is proportional 
to straight-line distance. 
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(a) [Q] Formulate this location-allocation problem 
as a constrained NLP. 


(b) K&] & Use class optimization software to start 
from reasonable locations and compute at least 
a local optimum. 


14-8 A light manufacturing firm is planning a new 
factory in arural part of the western United States. A 
total of 100 employees are to be hired from the 5 sur- 
rounding communities. The following table shows 
the number of (equally) qualified workers available 
in each community and community location coordi- 
nates (in miles). 


Community 
1 2 3 4 5 
Available Om al a2 0 40a 30 


E-W coordinate 0 10 6 1 2 
N-S coordinate 0 il 8 9 3 


Planners want to choose a factory site that mini- 
mizes total employee travel distance. Assume that 
travel distance is proportional to straight-line dis- 
tance from community to factory site. 


(a) Formulate this location-allocation problem as a 
constrained NLP. 

(b) [| & Use class optimization software to start 
from reasonable locations and compute at least 
a local optimum. 


14-9 An investor has decided to divide his $1.5 mil- 
lion portfolio among government bonds, interest- 
sensitive stocks, and technology stocks because some 
of these categories tend to increase return in periods 
when the others decrease. Specifically, his analysis 
of recent experience produced the following average 
returns and covariances among categories: 


Interest- 
Sensitive Technology 
Bonds Stocks Stocks 
Mean return 5.81% 10.97% 13.02% 
Covariance 

Bonds 1.09 1.12 3.15 
Interest- 1.12 iy 4.38 
sensitive stocks 
Technology -3.15 4.38 12.95 
stocks 


The investor wants to find the least variance way to 
divide his portfolio among the three categories while 
maintaining a 10% average return. 


(a) [§] Formulate this portfolio problem as a con- 
strained NLP. 

(b) [| & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-10 A farmer wants to allocate between 10 and 
60% of his available acreage to each of corn, soy- 
beans, and sunflowers. With markets varying wildly 
from year to year, he has done some research on 
past performance to guide his decisions. The fol- 
lowing table shows the average return per acre 
and the covariances among categories that he has 
computed. 


Com Soybeans Sunflow 
Dollar return 77.38 88.38 107.50 
Covariance 
Corn 1.09 -1.12 -3.15 
Soybeans -1.12 52, 4.38 
Sunflow -3.15 4.38 1295 


The farmer wants the least risk plan that will average 
at least $90 per acre. 


(a) Formulate this portfolio problem as a con- 
strained NLP. 

(b) EK] & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-11 A new premium whiskey will be produced by 
blending up to 5 different distilling products, and the 
quality of the results will be measured by 3 perfor- 
mance indices. The following table shows the value 
of each index for the 5 ingredients, along with lower 
and upper limits for the index of the blend and costs 
per unit volume. 


Ingredient 
Index 1 Ps 3 4 5 
1 12.6 S890 oesl 7.2, 10.1 11.7 
” 31.4 30.2 296 404 289 
5 115 202 184 143 169 


Cost 125 154 116 189 132 


Exercises 877 
Blend 
Index Low High 
ih 1D) 16 
2 3il 36 


3 121 164 


The index of the blend in the first two cases will be 
just a volume-weighted sum of those of the chosen 
ingredients. However, the third index is logarithmic 
(i.e., the natural logarithm of the blend value will 
be the logarithm of the volume-weighted ingredient 
sum). Production planners want to choose a blend 
that meets upper and lower index requirements at 
minimum cost. 


(a) [Q] Formulate this blending problem as a con- 
strained NLP. 

(b) [| & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-12 A chemical manufacturer needs to produce 
1250 barrels of a special industrial cleaning fluid by 
blending 5 available ingredients. The quality of the 
result is measured by 3 quantitative indices. The fol- 
lowing table show the index values for each ingredi- 
ent, along with the minimum and maximum required 
in the blend and the cost per barrel of ingredients. 


Ingredient 
Index 1 2 3 4 5 
1 50.4 45.2 33.1 29.9 44.9 
2 13.9 19.2 18.6 25.5 10.9 
3 89.2 75.4 99.8 84.3 68.8 
Cost 531 339 128 414 307 
Blend 
Index Minimum Maximum 

1 33 43 

Z 17 20 

3 81 99 


The index of the blend in the first case will be just a 
volume-weighted average of those of the chosen in- 
gredients. However, the square of the second blend 
index will be the square of the volume-weighted 
ingredient average, and the logarithm of the third 
blend index will be the logarithm of the volume- 
weighted ingredient average. Production planners 
want to choose a blend that meets upper and lower 
index requirements at minimum cost. 
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(a) Formulate this blending problem as a con- 
strained NLP. 

(b) [&] & Use class optimization software to start 
from a feasible solution and compute at least a 
local optimum. 


14-13 A laser printer manufacturer can make models 
i=1,...,6at any of plants; = 1,...,4. The frac- 
tion of plant j capacity required per unit of printer i 
has been estimated for each combination at f;;. The 
laser printer market is very competitive, so the price 
that can be charged for any model is a decreasing 
nonlinear (demand) function p; of the total number 
of model i printers that are sold. Assuming these de- 
mand functions are know, formulate an NLP to de- 
termine a maximum total revenue production plan. 


14-14 A new automatic storage and retrieval 
(ASAR)° area is being added to an existing ware- 
house on land already owned by the company. It 
will have n > 1 aisles, each with pallet storage cells 
on both sides and a stacker crane moving in the mid- 
dle which can carry a pallet to/from any location in 
the aisle. Storage must be provided for a total of at 
least p pallet cells of width w, depth d, and height h 
feet. All racks will be m > 1 cells high, and k cells 
from one end of the aisle to the other on each side. 
The total building height should not be more that ¢ 
feet, including a clearance of u feet between the top 
of the racks and the ceiling. Aisle width will be 150% 
of pallet depth to allow for clear passage of pallets 
carried by the cranes, and their length should accom- 
modate the k storage cells plus one extra pallet width 
to provide for an input/output station at the end. The 
ASAR area and its input/output stations will be en- 
closed with a roof and walls on three sides, being 
open only on the end (perpendicular to the aisles) 
where it adjoins the existing warehouse. Engineers 
want to find a minimum total cost design for the new 
facility using c; & unit cost of cranes, c. 4 unit cost 
of steel racks per pallet storage cell, together with 
3, C4, and cs being the construction cost per square 
of foundation/floors, ceilings, and sidewalls, respec- 
tively. Formulate an NLP model to choose an opti- 
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mal design in terms of decision variables n,m, k, and 
building exterior dimensions x (perpendicular to the 
aisles) y (parallel to the aisles) and z (height). Take 
all other symbols as constant. 


14-15 Assume that Syntex Laboratories’ is reexam- 
ining the distribution of its sales force across major 
pharmaceutical products j = 1,...,7. Present force 
levels e; are expected to produce s,; units of product 
sales per month at a profit margin of p,; per unit. 
However, extensive discussion and surveying has 
quantified impacts of changing the effort dedicated 
to different products. Nonlinear functions r;(x;) pre- 
dict the ratio of future to current product sales quan- 
tities as a function of x; A ratio of future to current 
sales effort. Formulate an NLP model in terms of 
decision variables x; to determine a maximum total 
profit realignment of salesforce distribution keeping 
the total force unchanged and the force devoted to 
each product within +50% of the current levels. 


14-16 A major oil company® manufactures 
petroleum lubricants at sites j = 1,...,10 using 
a critical additive purchased from suppliers i = 
1,...,15 all over the world. Manufacturing site j 
requires d; metric tons per month of the additive, 
and suppliers 7 can provide up to s; metric tons. The 
transportation cost for shipping additive from sup- 
plier 7 to manufacturing site j is a constant f;; per 
ton. However, the purchase cost from the supplier 
varies with volume. Starting with a base price c;, sup- 
plier i reduces the unit price for all purchases from 
i each month by a fraction a; > 0 for each multiple 
of quantity g; ordered. For example, the reduction 
might be fraction 0.5% = 0.005 off the base price 
for each 1000 metric tons ordered per month. For- 
mulate an NLP model to find a minimum total cost 
procurement and shipping plan using the decision 
variables (i = 1,...,15;7 =1,..., 10) 


x;; 4 metric tons shipped from supplier i to site j 


y; 4 total metric tons purchased from supplier i 


14-17 A stirred tank reactor? is a tank equipped with 
a large stirring device that is used in the chemical and 


Based on J. Ashayeri, L. Gelders, and L. Van Wassenhove (1985), “A Microcomputer-Based Op- 
timisation Model for Design of Automated Warehouses,” International Journal of Production Research, 


23, 825-839. 


7Based on L. M. Lodish, E. Curtis, M. Ness, and M. K. Simpson (1988), “Sales Force Sizing and 
Deployment Using a Decision Calculus Model at Syntex Laboratories,” Interfaces, 18:1, 5-20. 

8Based on P. Ghandforoush and J. C. Loo (1992), “A Non-linear Procurement Model with Quantity 
Discounts,” Journal of the Operational Research Society, 43, 1087-1093. 

°Based on L. Ong (1988), “Hueristic Approach for Optimizing Continuous Stirred Tank Reactors 
in Series Using Michaelis-Menten Kinetics,” Engineering Optimization, 14, 93-99. 


biochemical industry to produce chemical reactions. 
A series of 5 such tanks will be used to lower the con- 
centration of toxic chemical from co at input for tank 
1, to no more that ¢ on exit from tank 5. A flow rate 
of q liters per minute will be maintained through the 
tank sequence, but the effect of each tank depends 
on its volume v;. In particular, the units of toxic 
chemical removed per minute will be approximately 


output concentration 


1 + output concentration 


times the tank volume. Stirred tank cost can also 
be estimated from the volume at o(volume)*. For- 
mulate an NLP model to find a minimum total cost 
sequence of tanks using the decision variables (i = 
1a AS) 


c; & output concentration of tank i 


v; & size of tank i 


Assume that all other symbols are constants. 

14-18 Each day q; tons of freight arrive by sea!® in 
Japan bound for in-country regions 7 = 1,...,50. 
These goods may arrive at any of the major ports 
j = 1,...,17, but the internal transportation cost 
per ton c;; varies by port and destination. The 
government plans a capital investment program in 
port facilities to pick daily tonnage processing ca- 
pacities at each port j that minimize these inter- 
nal transportation costs plus associated port main- 
tenance costs, plus delay costs from freight passing 
through each port. Port j maintenance costs can 
be expressed a;(capacity i), where a; and b; and 
known constants. Delay cost at j can be estimated 
d/ (capacity j) — (traffic through j)], where d is the 
delay cost per ton per day. Formulate an NLP model 
to optimize the ports using the decision variables 
(i=1,...,50;7=1,...,17) 


x;; 4 tons shipped through port j for z 
x; A total tons shipped through port j 
y; & capacity of port j 
14-19 Three urban neighborhoods are mutually con- 
nected by freeways admitting traffic in both direc- 


tions. Net output b;, (per hour) at each neighbor- 
hood k of vehicles originating at i can be estimated 
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from known patterns (bi; = — )°,4; bi). The de- 
lay vehicles experience on any arc (i, j) is an increas- 
ing nonlinear function d;; of the total flow on that 
arc reflecting the number of lanes and other char- 
acteristics of the road link. Formulate an NLP to 
compute a “system optimal” traffic flow (1.e., one 
that minimizes the total delay experienced in carry- 
ing the required traffic), using the decision variables 
Gie= 1,8) 


Xij,4 & flow on arc (i, j) bound for node k 


(Observe that this may not be the same as a “user 
optimal” flow where each driver tries to minimize 
his or her own delay.) 


14-20 The commander of a battlefront must plan 
how to employ his f frontline and r reserve fire- 
power to minimize the advance achieved over days 
t = 1,...,14 by an attack of opposing forces with 
firepower a. Intelligence and battle simulations pre- 
dict that each surviving unit of firepower in the at- 
tacking force will kill p units of defender firepower 
per day, and each unit of defender firepower com- 
mitted to the battle will kill g units of attacker per 
day. Also, the kilometer advance on day ft can be 
estimated in terms of the ratio of forces fighting that 


day as 

defender firepower 4 

exp | —4.| ————__—_—_ 

attacker firepower 
Reserves will not be in the battle on day 1, but 
they may be committed as desired over the ensu- 
ing days. Once committed, reserves cannot be with- 
drawn from the battle. Formulate an NLP model to 


choose an optimal plan for the defender using the 
decision variables (¢t = 1,..., 14) 


x, A attacker firepower fighting on day t 
y, £ defender firepower fighting on day t 
z, & defender reserve firepower newly committed 
on day t 
Assume that forces are only lost to enemy fire. 


14-21 Chilled-water building cooling systems’? op- 
erate as indicated in the following sketch. 


10Based on M. Noritake and S. Kimura (1990), “Optimum Allocation and Size of Seaports,” Journal 
of Waterway, Port, Coastal and Ocean Engineering, 116, 287-299. 

UBased on K. Y. K. Ng and M. N. Lam (1995), “Force Deployment in a Conventional Theatre-Level 
Military Engagement,” Journal of the Operational Research Society, 46, 1063-1072. 

12Based on R. T. Olson and J. S. Liebman (1990), “Optimization of a Chilled Water Plant Using 
Sequential Quadratic Programming,” Engineering Optimization, 15, 171-191. 
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chiller 


condenser H4 


evaporator 


pump 


building — 


Water flows at a rate of F; gallons per minute around 
the lower loop, entering the chiller at temperature 
T,,1 and being cooled to temperature T; before 
passing through the building. An upper loop flow- 
ing at rate F, gallons per minute absorbs heat within 
the chiller in a separate water stream, and passes it 
through an outdoor cooling tower to reduce its tem- 
perature from 7,1 to T).. Tower output tempera- 
ture T>» is a nonlinear function f, of T>,, F, the 
tower fan speed S, and the ambient air temperature 
Ty. The lower loop must absorb a load H of heat 
within the building (measured in water temperature 
difference times flow). Similarly, the heat exchanged 
between the two loops should balance within the 
chiller after adjusting for extra heat generated by 
the unit and its pumps, which is a constant multi- 
ple k of the electrical energy they consume. That 
energy is, in turn, a nonlinear function f) of all four 
temperatures and both flow rates. Energy consumed 
in the cooling tower is another nonlinear function f; 
of fan speed. On any given day, temperature Tp, 
heat load H, constant k, and all 3 functions will be 
known, but other system variables can be controlled 
within ranges [T;,,, T;;], [F;, F:], and [S, S], respec- 
tively. Formulate an NLP model to decide how to 
operate such a system at minimum total electrical 
consumption. 


14-22 The figure below shows a system of reservoirs 
and hydroelectric dams of the sort operated by large 
utilities such as California’s PG&E.% 


Chapter Fourteen e Constrained Nonlinear Programming 


¥ 
Lak 


Each node is a reservoir with a power plant releasing 
water on the downstream side. The water then re- 
quires one month to reach the reservoir of the next 
dam on the river system. Acre-feet inflows b;, from 
streams feeding each reservoir i (other than the in- 
dicated rivers) can be estimated for coming months 
t=1,...,3, and reservoirs begin month 1 with stor- 
age S;o. Lower and upper bounds s, and 5; restrict the 
number of acre-feet of water that should be stored 
in each reservoir i at all times, and similar bounds ub 


< 


and f, limit the release flow through the dams. The 
hydropower obtained from any dam i in any month 
can be estimated by nonlinear function h;(s, f) re- 
flecting both the flow f through the dam that month 
and the water pressure, which is a consequence of the 
amount of water s stored in the corresponding reser- 
voir at the end of the month. Managers would like 
to operate the system to maximize the total power 
produced. 


(a) Sketch a time-expanded flow network for this 
problem with arcs f,, reflecting monthly release 
flows at i, and other arcs s;, reflecting ending 
storage there at month t. Label arcs with lower 
and upper bounds, and nodes with net inflows. 
Assume that all releases f,9 = 0. 

(b) Formulate an NLP to maximize the total power 
produced in your network of part (a). 


14-23 Determine whether each of the following 
NLP’s is a convex program. 


(a) E] max In(x,) +3x2 
St x, >1 
2x4 + 3X2 = 1. 
(41)? + G2)? < 9 
(b) min x,+%) 
s.t. X4xX2 < 9 
—5<x, <5 
—5<x% <5 


13Based on Y. Ikura, G. Gross, and G. S. Hall (1986), “PG&E’s State-of-the-Art Scheduling Tool for 


Hydro Systems,” Interfaces 16:1, 65-82. 


(c) Bl max x, +6/x, +5(x2)? 
s.t. 4x; + 6x2 < 35 
x, = 5) x2 => 0 
(d) min 14x, + 9x2 — 7x; 
s.t. 6x, + 2x2 < 20 
3x2 + 11%; < 25 
X11, %25;X%3 = 0 
(ce) BR) min et — 28x, 
St. “Ga —3)@-- Go — 5)* <4 
14x, ee 6x2 =i Le) 
—2(x1)? + 2x1x2 — (x2)? = 0 
(f) max 62x, + 123x2 
s.t. In(x;) + In(x2) = 8 
7x; + 2x2 < 900 
X1,Xe = al 


14-24 [6] Determine which of the NLPs in Exercise 
14-23 are separable programs. 

14-25 Determine whether each of the following 
NLPs is a quadratic program, and if so, identify the c 
and Q of matrix objective function form e-x + xQx. 


(a) &) min xx +134/x3 + In) 
s.t. xy + 4x aay 7 
14x, + 2x3 = 16 
X1, X2,%3 = 0 
12(x,)? + 34x4x2 + 5 (xo)? 
—12x, + 19x 
st.  7xj + 3x2 > 15 
93x, + 27x2 + 11%; < 300 
x32 0 
(c) KB] max 2x,x2 + (%)* + 9x3 
s.t. xX, +x. +%3 > 6 
30 << 5) 57] eens) 
(d) max (x1)? + (2)? 
s.t. (x4 — 10)? + (x - 4)? <9 
X1,%. > 0 


(b) min 


14-26 Determine whether each of the following is a 
posynomial. 


(a) [O] 23x, — 34x. + 60x; 

(b) 54x; + 89x. + 52x3 

(c) [Q] 7xix2/(03)?? + 4.01 

(d) 44x;/In(x2) +e 

14-27 Demonstrate that each of the following NLPs 
is a posynomial geometric program by placing the 
model in standard form and detailing the sets K;, 
and associated coefficients d;, and a,j. 


(a) [min 13x1x2/x3 + 9./11%3 
st. 3x, + 8x2 < x3 
20/(x3)* <4 


X1, X2, X3>0 
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(b) min 40/21 + X2/./X3 
S.t. XxX Ss (x3)” 

18x, + 14x. <2 
X1,%X%2,X3 >0 


14-28 Consider the nonlinear program 


min 8(x, —2)? +2(x, —1)* 
s.t. 32x + 12x = 126 


(a) [Q] Form the Lagrangian function for this 
model. 

(b) 3] Write stationary conditions for the La- 
grangian. 

(c) [&] Solve your stationary conditions for x; and 
X2, and explain why your answers are optimal in 
the original model. 

(d) Explain why a constraint x; < 2 would be active 
if added to the original model. 

(e) [Q] Use the Lagrangian approach to compute an 
optimal solution to the model with the added 
constraint of part (d). 


14-29 Do Exercise 14-28 for the NLP 
max 300 —5(x; — 20)? — 4(x, — 6)? 
s.t. x +x. =8 


and part (d) extra constraint x2 > 0. 


14-30 State the Karush-Kuhn—Tucker optimality 
conditions for each of the following mathematical 
programs. 


(a) B)] min 14(, — 9)? +3(2 — 5)? + (x3 — 11)? 
st. 2x, + 18x. — x3 = 19 
6x; + 8x2 + 3x3 < 20 
X1,X2 = 0 
(b) max 6x, + 40x. + 5x3 
s.t. X1 Sin(x2) + 9x3 < 2 
elrit3e2 4 14x, > 50 
XI Xi = 0 
(c) Key min 100 (x1 3)? (x2)4 t 19x3 
st: . S@q— 1)? + 30 (ay 22) S385 
60x. + 39x3 = 159 
X41, %X2,%3 = 0 
(d) max 7In(x1) + 4In(2) + 11 In) 
st. (xy +2)? — xyx2 + (%2 — 7)? < 80 
5x1 + 7x3 — 22 
X1,%2,%3 = 1 


14-31 [6] For each mathematical program in Exer- 


cise 14-30, determine whether principle | 14.26] as- 
sures that a KKT point is a global optimum. 
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14-32 Consider the NLP 
min 15(x)? = 4(x2)? 
s.t. 3x1 + 2x2 =8 
X1,X = 0 


(a) [Q] State the KKT optimality conditions for this 
model. 

(b) Verify that at solution x = (0, 4) there exists an 
improving feasible direction Ax = (2, —3). 

(c) Confirm that KKT conditions have no solution 
for the nonoptimal x of part (b). 

(d) Explain why every local optimum of the model 
must be a KKT point. 

(ce) &] Show that global optimal solution x* = 
(1, 3) isa KKT point. 


14-33 Do Exercise 14-32 for NLP 
max 2In(x,) + 8In(x2) 
s.t. 4x, +x =8 
X1,%2 > 1 


with nonoptimal point x = G, 2), improving fea- 
sible direction Ax = (—1, 4), and global optimum 
x Cle 4): 

14-34 [9] Use absolute value (unsquared) penalty 
functions to reduce each NLP of Exercise 14-30 to 
an unconstrained penalty model. 


14-35 [Q] Do Exercise 14-34 using squared penalty 
functions. 


14-36 Consider the NLP 
min 2(x, — 3)? —xyx_ + (x — 5)? 
st. (a1)? + (x2)? < 4 
O0<x, <2, x%>0 


with optimal solution x* = (1.088, 1.678). 


(a) EK] Use unsquared penalty functions to re- 
duce this problem to an unconstrained penalty 
model. 

(b) [6] Explain why local minima of the uncon- 
strained model in part (a) must be global minima 
for all > 0. 

(c) [| Determine whether the penalty objective of 
part (a) is differentiable. Explain. 

(d) [Q] Determine whether there will be a penalty 
multiplier j large enough that the uncon- 
strained optimum in part (a) is optimal in the 
original model. Explain. 

(e) ] Suppose that we are solving the given con- 
strained NLP by the sequential unconstrained 
penalty Algorithm 14A. Explain why it is rea- 
sonable to begin with multiplier 4 = 0.5 and 
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increase it by a factor 6 = 2 after each uncon- 
strained optimization. 

(f) [| 2 Use class optimization software to apply 
Algorithm 14A, starting at x® = (3,5) and 
managing the penalty multiplier as in part (e). 


14-37 [S| Do Exercise 14-36 using squared penalty 
functions. Stop the search in part (f) when the total 
constraint violation is < 0.2. 


14-38 Do Exercise 14-36 for the NLP 


max 100 — 8(x,)? —3(x%) — 3)? 
S.t. xX. = 2/x1 

0O< x, <2 

0< X2< 2 


Start at x = (2,2) with multiplier ~ = 0.5, and 
increase by the factor B = 4. 


14-39 Do Exercise 14-38 using squared penalty func- 
tions. Stop the search in part (f) when total con- 
straint violation < 0.2. 


14-40 [6] Determine whether barrier methods can 
be applied to each of the NLPs in Exercise 14-23, 
and if so, use log barrier functions to reduce the 
constrained optimization model to an unconstrained 
barrier model. 


14-41 [] Do Exercise 14-40 using reciprocal barrier 
functions. 


14-42 Consider solving the NLP of Exercise 14-36 
by barrier methods. 


(a) [&] Use logarithmic barrier functions to reduce 
this problem to an unconstrained barrier model. 

(b) [| Explain why local minima of the uncon- 
strained model in part (a) for all ~ > O must 
be global minima. 

(c) EQ] Determine whether the barrier objective of 
part (a) is differentiable. Explain. 

(d) [&] Determine whether there will be a barrier 
multiplier 4 > 0 small enough that the uncon- 
strained optimum in part (a) is optimal in the 
original model. Explain. 

(e) [| Suppose that we are solving the given con- 
strained NLP by the sequential unconstrained 
barrier Algorithm 14B. Explain why it is rea- 
sonable to begin with multiplier ~ = 2 and de- 
crease it by a factor B = 4 after each uncon- 
strained optimization. 

(f) | & Use class optimization software to apply 
Algorithm 14B, starting atx© = (3, 5) and man- 
aging the barrier multiplier as in part (e). Pro- 
ceed while u > 3. 


14-43 [O] Do Exercise 14-42 using reciprocal barrier 
functions. 

14-44 Do Exercise 14-42 for the NLP of Exercise 
14-38. Start atx = (1.8, 1.8) with multiplier w = 8, 
and decrease with factor B = §. 

14-45 Do Exercise 14-44 using reciprocal barrier 
functions. 


14-46 Consider the nonlinear program 


min (x; — 8)? +2(x2 — 4)” 
st. 2x, + 8x. < 16 

xy <7 

X1,X%2 = 0 


(a) [Q] Introduce slack variables x3 and x4 to place 
the model in standard form for reduced gradient 
Algorithm 14C. 

(b) [S| Show that x2 and x4 form a basic set of vari- 
ables for your standard form. 

(c) [| Assuming the basis of part (b), classify vari- 
ables as basic, nonbasic, or superbasic at initial 
solution x = (0, 1, 8, 7). 

(d) [] Compute the reduced gradient correspond- 
ing to the basis and x of part (c). 

(e) [Q] Construct the move direction that would be 
pursued by Algorithm 14C at the basis and x”) 
of part (c). 

(f) [| Compute the maximum feasible step in the 
direction of part (e). Then, assuming (correctly) 
that the direction of part (e) remains improving 
all the way to the maximum A, compute the re- 
sulting new solution x. 

(g) [|] Explain why a basis change would by re- 
quired by Algorithm 14C at the x" of part (f). 
and choose an appropriate new basis. 


14-47 Return to the standard form NLP of Exercise 
14-46(a). 


(a) EQ] Apply reduced gradient Algorithm 14C to 
compute an optimal solution starting from the 
x = (0,1, 8,7). 

(b) Graph your progress in a plot of the feasible 
(1, 2). 


14-48 Do Exercise 14-46 for nonlinear program 


max 500 —3(x; +1)? + 2x1x2 — (x2 — 10)? 
S.t. xy —-X. <1 

x= 5 

1, Xo = 0 


using basis {x;, x4} and standard-form starting solu- 
tion x = (2, 1, 0, 4). 
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14-49 | Do Exercise 14-47 on the standard-form 
NLP of Exercise 14-48(a). 


14-50 Consider the equality-constrained quadratic 
program 
min  6(x1)? + 2(%)? — 6xyx2 + 4(x3)? 
+ 5x; + 15x. — 16x3 
st. x, + 3x2 — 2x3 = 2 
3x1 —%. +4x3=3 


(a) [J] Identify the Q, c, A, and b of (symmetric) 
quadratic program standard form. 

(b) [§] State Karush-Kuhn—Tucker optimality con- 
ditions for the model as a system of linear equal- 
ities. 

(c) [| & Solve your system of part (b) for the 
unique KKT point of the model. 


14-51 Do Exercise 14-50 for the equality-constrained 
quadratic program 


— (x1)? — 8(%2)? — 2(x3)? + 10x23 
+ 8x, — 14x. + 20x3 

S.t. xX, +4x3 =2 

—x2+3x3=1 


max 


14-52 fi Return to the NLP of Exercise 14-46, and 
consider solving by active set Algorithm 14D start- 
ing from solution x© = (0, 1). 


(a) [6] Demonstrate that the model is a quadratic 
program by deriving the co, ¢, Q, a®,..., a, 
b;,...,b4, G, L, and E of general symmetric 
form [14.50] ' 

(b) [| State and solve as a system of linear 


equalities the active set optimality conditions 
| 14.53 | corresponding to initial solution x. 


(c) [&] Determine the step 4 that would be applied 
to the direction resulting from part (b), and com- 
pute the new point x. 

(d) [| Verify by forming and solving the optimality 


conditions | 14,53 | corresponding to x"? that no 
further progress can be made if all inequalities 
active at x") of part (c) are included in the active 
set S. 

(ec) [| Use the results of part (d) to show which ac- 
tive constraint should be dropped from S. 

(f) [&] Begin from part (e) and complete the solu- 
tion of this quadratic program. 

(g) Graph your progress in a plot of the feasible 
(%1, X2). 
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(h) Compare the evolution of active set Algorithm 
14D in part (g) with corresponding reduced gra- 
dient Algorithm 14C computations in Exercise 
14-47. 


14-53 ] Do Exercise 14-52 for the NLP of Exercise 
14-48 starting from solution x = (2, 1). 


14-54 Form linear programming approximations 


14,59 | to each of the following separable programs 


using breakpoints uw = 0, m1 = 1, 2 = 3, 
42,0 = 0, U2, = 25 u2.2 = 4, 


(a) KE) min x1/(4-—x1) + @ — 1)? 
s.t. 2x, +x. >2 
4(x, + 1)? — 91)? < 25 
O<x, <3 
0<%<4 
(b) max 500- (x, — 1)? —25/(x +1) 
st. Jp — (2 +1)? > -3 
6x, + 2x2 < 10 
O0<x, <3 
0<x% <4 


14-55 Consider the trivial separable program 


min 2(x — 3)? 
st. O<x <6 


(a) Verify that the model is a convex program. 

(b) [&] Verify by inspection that an optimal solution 
occurs at x* = 3. 

(c) [| Form a linear programming approximation 


14°59 | using up = 0, uy = 2, uy = 6. 


(d) [] Solve your LP approximation of part (c) by 
inspection, and determine whether correctness 
condition is satisfied by the approximate 
optimum. 

(e) Discuss how convexity of part (a) relates to cor- 


rect sequencing| 14.60 | in part (d). 


(f) Verify by inspection that x* = 0 and x* = 6 are 
alternative optima in the original NLP when the 
objective is maximized instead of minimized. 

(g) EQ] Repeat part (d), this time maximizing the 
objective function. 

(h) Comment on the errors introduced in objective 
function and other values when sequencing con- 


dition is violated in part (g). 
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14-56 Consider the standard-form posynomial geo- 
metric program 


min 3/./%; +x1X2 + 10/(x3)° 

st. 0.5x1x2/(x3)? <1 
0.167x, + 0.25(x1)°4x2 + 0.0833x3 <1 
X1,X%2,X%3 > 0 


(a) [&] Change variables to convert this geometric 
program into a convex program. 

(b) [| & Use class optimization software to solve 
your convex program of part (a) and transform 
optimal variable values back to obtain an opti- 
mal solution for the original NLP. 

(c) [| Form the geometric programming dual of 
the original NLP. 

(d) [| Determine the degree of difficulty of the 
original NLP. 

(e) | & Use class optimization software to solve 
the dual of part (c) and retrieve an optimal pri- 
mal solution from the corresponding Lagrange 
multipliers. 


14-57 Do Exercise 14-56 for the posynomial geomet- 
ric program 


min 10/(x1x2x3)? 

st. 12(x1)?x. + 4x3 <1 
0.1%2,/x1 + x2X3 al 
(44x,)028 <1 
X1,X2,x3 > 0 


14-58 A water distribution system“ is a network with 
(positive = forward or negative = reverse) flows x;; 
in pipes between nodes i,j = 0,...,m represent- 
ing storage tanks and pipe intersections. Pressures 
at the nodes 7 can be measured in hydraulic “head,” 
whichis the height to which water will rise in an open- 
ended vertical pipe installed at the node, relative to 
the “ground node” 0. Heads have assigned values 
8; for storage tank nodes i, and net outflows r; are 
established for all nodes (}°7".) r; = 0). The ground 
node 0 is connected to each storage node i by an arc 
(0, i), and to no others. To determine how the sys- 
tem will perform at steady state, engineers need to 
find flows f,; and heads ; that (i) maintain net flow 
balance at every node, (ii) achieve assigned heads s; 
at storage nodes, and (iii) satisfy nonlinear head-to- 
flow equations 


hp lh Org (ig) 


nonground (i, j/) 


M4Based on M. Collins, L. Cooper, R. Helgason, J. Kennington, and L. LeBlanc (1978), “Solving the 
Pipe Network Analysis Problem Using Optimization Techniques,” Management Science, 24, 747-760. 


where functions ¢;;(x;;) are known relations be- 
tween head difference and the flow on particular arcs 
(i,j) that reflect length, size, pumping, grade, and 
other characteristics. 


(a) Formulate a related NLP over unrestricted 
flows x;; having only flow balance constraints 
at all nodes and a minimizing objective function 
summing terms 


SjX0,j arcs (0, J) 

Fi Qi) xg, = 

0” Gij(z) dz other (, j) 
SUGGESTED READING 


Bazarra, Mokhtar S., Hanif D. Sherali, and C. M. 
Shetty (1993), Nonlinear Programming Theory and 
Algorithms, Wiley, New York. 
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(b) Explain why your NLP of part (a) is a separable 
program. (With mild assumptions on the ¢j; it 
can also be shown to be convex.) 

(c) State Karush-Kuhn-Tucker conditions for the 
primal model of part (a) and explain why they 
must be satisfied by a locally optimal x*. 

(d) Interpret conditions of part (c) to show that 
a solution to steady-state equation system (i)- 
(iti) above can be obtained from locally optimal 
flows x* in part (a) and corresponding KKT mul- 
pliers v*. 


Luenberger, David G. (1984), Linear and Nonlinear 
Programming, Addison-Wesley, Reading, Mass. 


Selected Answers 


Chapter I 


1-1 (a) s (b) d, p, and b (ce) min(d/s)* (d) ps < b,s 
nonnegative and integer 

1-2 (a) feasible and optimal (b) neither because 
infeasible (c) feasible not optimal 

1-5 (a) exact numerical optimization 

(c) closed-form optimization 

1-8 (b) 16.4 hours; 166.1 days; 110.6 years; 

6.5 million years 

1-9 (a) random variable (c) deterministic 

(e) deterministic (g) random variable 

(i) deterministic 


Chapter 2 


2-1 (a) max 200x, + 350x2, s.t. 5x; + 5x2 < 300, 
0.6x, + 1.5x2 < 63, x, < 50, x2 < 35,x, > 0,x. > 0 
(b) x} = basic = 30, x} = deluxe = 30 (d) all optimal 
from x = (30, 30) to x = (17.5, 35) 

2-2 (b) x} = domestic = $5 million, x} = foreign = 
$7 million 


2-3 (b) x} = Squawking Eagle = 40 thousand, x} = 
Crooked Creek = 10 thousand 

2-4 (b) x} = beef = 25g, x5 = chicken = 100 g 

2-5 (b) v* = 7000, c* = 0 

2-6 (a) min x, + X2,S.t. Sx, + 3x2 > 15, 

2x, + 5x2 > 10,0 < x, < 4,0 <x < 4,1, x integer 
(b) partial patterns make no physical sense 

(c) Either xf = x} = 2, or xf =3,x5 =1 

2-7 (a) min 16x; + 16x2, s.t. x1x2 = 500, x; = 2x2, 
X2 < 15, x; = 0, x. > 0 (b) x} = length = 33; feet, 
x} = width = 15 feet (d) x; < 25 leaves no feasible 
2-8 (b) x} = diameter = 78.16 feet, x; = floors = 
31.26 


2-9 (a) max w, (b) max 5w + 2w2 (c) max w2 


2-11 (a) min >, ee yi (ce) max SYP, ayia 
OLadontas tS hes 

2-12 (a) 7), xij < 200,7 =1,...,5,t=...,7;35 
constraints (b) ae ae X5,j. < 4000; 1 constraint 
(©) hy Mie = 100, = 1,...,17;¢=1,...,75119 


constraints 

2-16 (a) f(v1, yo, y3) 2 (11) y2/ys, 
81/1, Yo. V3) & 1 + y2 + y3, 

by = 13, go(y1, yo, v3) & 2y1 — yo + Oya, 

bp = 0, g3(01, v2, ¥3) 2 yi, 

b3 = 0, g4(y1, yo. ¥3) 2 y3, bs = 0 

2-17 (a) linear (c) nonlinear (e) nonlinear 

(g) nonlinear 

2-18 (a) LP (c) NLP 

2-19 (a) continuous (ce) discrete 

2-20 (a) ai Xj =3 (c) x3 +xg <1 

2-21 (a) max 85x; + 70x2 + 62x3 + 93x4, s.t. 
700x, + 400x2 + 300x3 + 600x4 < 1000, 

xj =Oorl, j=1,...,4 @®) fund 2 and 4 (ie., 
xf = x5 = 0, x5 = x4 = 1) 

2-22 (b) build 2 and 4 (i.e., ys = ys = 0, 
yaa D 

2-23 (a) ILP (c) INLP (e) INLP 

2-24 (a) model (b) (c) model (d) 

2-26 (b) nonzeros: xf = 1000, x}, = 15,000 
2-27 (i) xf = x5 = x3 = 1100, x7 = x7 = 1500, 
x= = 1400, x7 = 400, xg = x}, = 0, x5 = 1900 
2-28 (h) xf =x} = xt =x = 1 

2-29 (h) nonzeros: x7, = 81, x}, = 93, x7, = 166, 
xt, = 90, x¢, = 85, xt, = 145, x8. = 301, 

xf, = 166, x3, = 105, x3, =99 

2-32 (g) x5 =x} = xj = 1, others = 0 

2-34 (h) nonzero values: xj, = 115, x4; = 165, 
xt, = 85, xt, = 225 
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2-36 (i) x}, = 50, xj; = 25, x3) = 5, yj = 5000, 
y3 = 8500 

2-38 (i) nonzeros: 
34 


xss=lys=y3=ys=l 


ae oo a 
%72 = 2,4 X33 


Chapter 3 


3-1 (a) feasible and local max; infeasible; feasible; 
feasible, local and global max 

3-2 (a) y = (8, —2, 5), y? = @G, 8, 10), 

y® = (3, 11, 10) 

3-3 (a) Aw” = (4, —2, 6), AW® = (0, —2, 8), 
Aw®) = (-1, 0,3) 

3-4 (a) nonimproving (c) improving 

(e) nonimproving 

3-5 (a) feasible (c) feasible (e) infeasible 

3-6 (a) A = 3; not unbounded (ce) A = +00; 
unbounded 

3-7 (a) improving (c) need more information 

(e) improving 

3-8 (a) Aw = (3, —2, 0, 1) (c) Aw = (-8, 3) 

3-9 (a) [ii], [iv] 

3-10 (a) feasible (c) infeasible 

3-11 (a) 2 Aw; +3 Aw; = 0, 

1Aw,; +1Aw2+2 Aw3 =0, Aw; > 0 

(c) 1 Aw; +1 Aw2 = 0,2 Aw; — 1 Aw, = 0 

3-12 (a) (4,7) - (2,0) > 0, (4,7) - (-2,4) >0 
(b) Az® for A = 2 to z) = (4,0), Az for A = 3 to 
z® = (2,3), Az™ for A = § toz® = (3, 3) 

3-14 (a) unimodal (c) unimodal (e) not unimodal, 
x = (1,3), x® = (2, 1) (g) unimodal 

3-15 (a) (3, 1, 0) + A(—3, 3, 9), a € [0, 1]; 4 = i for 
Zz; no A gives 2 

3-16 (a) not convex, x = (0, 3), x®@ = (3,0) 

(c) convex (e) not convex, x = (0, 0, 0, 4), 

x? — (0,0, 0,5) 

3-17 (a) min w4 + Ws, s.t. 

40w, + 30w2 + 10w3 + w4 = 150, wy — wz < 0, 
4w2 + w3+Ws = 10, Wi, W2, W3, Wa, Ws = 0; 

w4 = 150, ws = 10 (c) min w3 + wg + Ws, St. 

(w; — 3)? + (W2 — 3)? — w3 < 4, 2w) +2w. +w4 =5, 
W1 + Ws = 3, W3, Wa, Ws = 0; w3 = 14, w4 = 5, 

Ws = 3 

3-18 (a) stop and conclude model is infeasible 

(c) proceed with Phase IJ from initial solution 

y= (53.1) 


3-19 (a) max 22w, — w2 + 15w3 — M(w4 + ws), S.t. 
40w, + 30w2 + 10w3 + w4 = 150, wi — w2 < 0, 

Aw. + w3+ ws > 10, w1, Wr, W3, Wa, Ws > 0; 

w4 = 150, ws = 10 (c) min 

2w; + 3w2 + M0w3 + wa + ws), St. 

(Ww 3 3y2 + (w2 — 3) —w3 <4,2w, +2w.+w4 =5, 
W, + Ws > 3, W3, Wa, Ws > 0; w3 = 14, we = 5, 

Ws = 3 

3-20 (a) stop and conclude model is infeasible if M 
is big enough, else increase M and repeat (ce) stop 
and conclude y = (1, 3, 1) is a local optimum for the 
original model 


Chapter 4 


4-1 (a) x; 4 cases shipped to region j, max 

1.60x, + 1.40x2 + 1.90x3 + 1.20x4, s.t. 

as = 1200, 310 < x, < 434, 245 < x. < 343, 
255 < x3 < 357, 190 < x4 < 266 (b) xj = NE = 408, 
x3 = SE = 245, x3 = MW = 357, x4 = W = 190 
4-2 (b) nonzero values are x}, = 70, x}, = 10, 

X32 = 40, x5, = 5,54 = 35,33, = 80 


4-3 (a) x; A fraction of ingredient j; min 

200x1 + 150x2 + 100x3 + 75x4, st. 2, x) = 1, 

60x; + 80x2 + 55x3 + 40x4 = 60, 

50x, + 70x2 + 40x3 + 100x4 < 60, 

90x, + 30x2 + 60x3 + 80x4 > 60 (b) the 3 main 
inequalities (c) xf = oats = 0.157, x; = corn 

= 0.271, x} = alfalfa = 0.401, x3 = hulls = 0.171 
4-4 (c) xj = 0.176, x5 = 0.353, x3 = 0.000, 

xj = 0.471 

4-5 (a) 45 0 Mig S Gi %ij S 48 Oy Xj, 
jHd,- WO Sys = 110 ye, rhs 

(e) 7x1, <3 es Xij,J = (Oy sud glil 

3 e jot Xij = Diet Deja Xi 

4-7 (a) x; 4 number of cuts with pattern j; min 
0.34x, + 0.22x2 + 0.27x3, s.t. 2x, + 1x3 > 37, 

5x2 + 3x3 > 211,x, >0, 7=1,...,4 (b) xf = 147.4, 
5 1010 0 Ae 

4-8 (b) x} = santas = 147.4, x; = trees = 0.0, 

x3 = houses = 21.1 

4-9 (a) max 30x; + 45x, st. 

0.30x1, + 0.30x2 + 0.10x3 + 0.15x4 + 0.50x5 < 80, 
1.5x3 +2.5x4 < 500, x3 = 4x1,x4 = 4x2,X5 =X, +X, 
x; = 0,7 =1,...,5 (b) the 3 equalities (c) xf = 27.8, 
> = 333 aq Tel a 838 ol 


4-10 (c) x} =0.0, x3 = 666.7, x3 = 0.0, x} = 2000, 
xt = 12,000 

4-11 (a) DS Xip = ai dip + pa ba eh GkXkps 
(At ene 

4-12 (a) x; 4 number with 5 days starting 
Sunday,..., x7 4 number with 5 days starting 
Saturday; 

min i OG 

St. 1 + X¥4 +X5 + X65 +x7 > 8, 

XM +2 +45 +%6 +x7 > 6, 

Hy + X2 +X3 +%6 +X7 = 6, 

X1 + X2 +X3 +4 + 2x7 = 6, 

xy + X2 + %3+%4+X5 > 6, 

x2 +X3 +X4+X5 +X => 10, 

X3+X4+X5 +X +X7 > 10, 

Been Oa ll ede 

(b) all main constraints (¢) x7 = 0.0, x5 = 0.67, 

x3 = 2.0, x} = 2.67, x5 = 2.0, x§ = 2.67, x3 = 0.67 
4-13 (¢) xf =5,x3 = Ohne = 0.25 = 0, x3 = 3, 

xf =3 


4-14 (a) x;, 4 investment in option j, year t; max 
1.05%1,4 + 1.12x23 + 1.21x31, st. 

10 = x11 +21 +31, 

1.05x11 + 10 = x12 +. x2, 

1.05x1,2 + 1.12x24 +10 =%134+%3, 

1.05x1,3 + 1.12x22 = x1,4, all variables nonnegative 
(b) all main constraints (¢) 4 years (d) nonzero 
values are x3, = x3, = 10, x3, =xt, = 21.2 

4-15 (d) x} = x3 = xj = 1200, x7 = 650, zi = 0, 

2) = 150, z3 = 1250, z} = 1600 

4-16 (a) 0, GinXit < De, K=1,..., 


(©) x1 = dit +21,i=1,...,m 


G,0 =e 


4-17 (a) wr & overestimation on point j, 

we 4 underestimation on point j; 

min = (wy + wy), 

st. Bo +26; =1+wt —wy, 

Bo + 3B; =3+w3 — wy, 

Bo +5f1 =3+wz — wz, 

Bo + 7B, =5+wi —wy, 

all variables nonnegative (b) 6* = 0.000, 

PB; = 0.714, nonzero over w{* = 0.429, wf* = 0.571, 
nonzero under w;* = 0.857 (c) 0.429, 0.857, 0.571, 
0.000 


4-18 (b) x* = 10, nonzero runs w}* = 15, w)* =5 
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4-19 (a) z A largest deviation, we A overestimation 
on point j, w; 4 underestimation on point j; min z, 
8.t. 

bo +26; =1+wf -—wy, 

Bo + 361 = 3+ wy — wy, 

Bo +561 =3+wy — wy, 

Bo + 7f1 =5+w7z — wy, 

z>wy, PAF ty ewe pe, A 

all variables nonnegative 

(b) 8) = 0.333, B* = 0.667, z* = 0.667, nonzero 
over w{* = wi* = 0.667, nonzero under 

wz = 0.667 (¢) 0.667, 0.667, 0.667, 0.000 

4-21 (a) min ae CjX;, Sut. pas GijXj = Vi, 
i=1,...,7,0 <x; <uj,j=1,...,5; where the aj, 
are the yield fractions in the table, c; the costs, and 
u, the availabilites; the r; are the given 
requirements. (b) x} = 18.75, x3 = 125.00, 

x3 = 150.00, x7 = 650.00, xt = 0.00 

4-22 (b) Nonzeros 

Xi, =P. =x}3 = 833.3, Xf 4 = 500.0, 

21 = X32 =%23 = 10,000.0, x3 , = 6000.0, 

X31 = 73.2 = 43,3 = 833.3, x}, = 500.0, 

X41 = X42 =%43 = 833.3, xf, = 500.0, 

X51 = X59 = XE, = 833.3, xf, = 500.0, 

hs, = 533.3, ht, = 166.7, hs , = 200.0 

4-23 (b) s| = 5; = 0,53 = 8, 55 = 12, s* = 56 = 24, 
§ = 32,4 =12,6=4 =8, f=4 = = 16, 

tf = 20 


Chapter 5 


5-1 (b) boundary and extreme, infeasible, interior, 
boundary not extreme (c) —w; + w> < 1 and W2 <3 
active; no active; w2 < 3 active (d) optimal or 
unique, neither, neither, optimal not unique 


5-3 (a) A= 9 0 6 0 0 |, 
=5. 9 0 Ogr=al 
b = (12, 15, 3), e = (4, 2, —33, 0, 0) (ce) 


2p i Ie at 0 0 0 

1 O° OF 250. 1 0 0 
— 0 1 0 0 0 il 0 | 

0 0 1 0 0 0 1 


b = (0, 3,3, 3), e = (15, 41, —11, 0, 0, 0, 0) 


890 


Selected Answers 


1 -1 1 dahl 
(e) A= ( 0 He acer a ),p= ao —6), 
c= Cal, 14.10) 
= ale 
5-4 ma sr ae | ).B=@.19 


(c) yes, no, yes, yes, no, no (d) y = (2, 4, 0, 0) 
feasible, y = (0, 0, 2, 10) feasible, y = (—2, 0, 0, 20) 
infeasible (e) y = (2, 4), y = (0, 0), y = (2, 0) 
5-6 (a) 5 Aw; + 1 Aw2 — 1 Aw3 = 0, 
3 Aw; —4Aw2+8Aw3 =0, Aw2 > 0 
5-7 (a) x = (1, 0,3, 0) (b) Ax = (—3, 1, —1, 0) for 
x2, Ax = (1,0, —5, 1) for x4 (ce) each has AAx = 0, 
Ax, => Oand Axg > 0 @) no, yes (e) A = 4, {x, x3}, 
d= 2, (x1, Xa} 
5-9 (a) zj =4, 75 =2 (bd) 
—2 1 1 0 0 
A= 1 il 0 1 0 |, 
1 0 0 0 1 
b = (2, 6, 4), c = (3, 1, 0, 0, 0) (c) bases {z3, z4, zs}, 
then either {z1, 23, Za}, {Z1, Z2, Z3} or {Z2, Za, Zs}, 
{Z1, Za 2s}, {Z1, Z2, 23} (d) z = (0, 0), then either 
Zz) = (4,0), 2% = (4,2) orz = (0, 2), 
7) = (4, 8), Zz = (4, 2) 
5-11 (a) yes (c) no (e) no 
5-12 (a) z = 10 — 29x. + 10x3, x; = 1 — (3x2 — 1x4), 
xX. = 3 — (1x. + 5x4) 
5-13 (a) no (c) yes 
—1 il 1 0 
1 -1 0 1 ) 
b = (4, 10), e = (4,5, 0, 0) (c) yo = (0, 0, 4, 10), 
then either y = (10, 0, 14, 0) and Ay = (1, 1, 0, 0) 
for y2 improves without limit, or y = (0, 4, 0, 14) 
and Ay = (1,1, 0, 0) for y, improves without limit 
5-16 (a) min we + w7,s.t. wy + w2 + wa = 18, 
2w, +w3— We = —2, 3w2 +5w3— ws +w7 = 15, 
W1,..-,W7 = 0; {W4, Wo, W7} 
5-17 (a) max 2w; + w2 +9w3 — M(we +7) s.t. same 
constraints and starting basis as in Exercise 5-16 


De cent tay. Ore aaa 
st A= ( (Onan welig: 2:0) 1 a 


b = (2, 1), mind = (0, 0,0, 0, 1) 

(c) y = (0, 0,0, 1,2), y® = (0,1, 0, 0, 1), 
optimal value positive 

5-20 (a) (4!)/(2!2!) = 6 (©) (2340!)/(1150! 1190!) 


5-14 (b) A= ( 


5-21 (a) yes (c) no 
5-22 (a) alternative optima x = (3, 6) through 
x = (6, 3) (b) 
1 1 il 0 0 
A=] —2 1 0 il 0 
1 -2 0 0 1 
e = (1,1, 0, 0, 0) (c) bases {x3, x4, x5}, then either 
{x1, x3, x4} and {x1, x2, x4}, or {x2, x3, x5} and 
{x1, X2, Xs} (d) x© =x = (0, 0), then either 
x® — (3,6) orx® = (6,3) 


,b = (9, 0, 0), 


0 0.500 


co el = 
ee ( 0.250 0.125 


(b) v = (0, 0.5) 


(c) no for x2, yes for x4 (d) E = ( A Pe ) 


0 0.2 
pu (0050. 0525 
~\ 0.050 0.025 


5-26 same as Exercise 5-9 
5-28 (a) nonimproving, 4 = 2, (L, B, B, L, B) 
(c) improving, A = 5, (L, B, U, B, B) 
5-29 (a) zj = 3, 23 =0 
1 1 -1 0 
(b) A= ( gp nga tS )w=a., 
ce = (5, 6, 0, 0) (c) basis statuses (U, U, 1st, 2nd), 
then either (L, U, 1st, 2nd), (L, 2nd, 1st, L), 
(1st, 2nd, L, L) and (1st, L, L, 2nd), or 
(U, L, 1st, 2nd) and (1st, L, L, 2nd) (d) z = (6,5), 
then either z? = (0,5), z® = (0, 4) z® = (2, 1) and 
z°) = (3,0), or z = (6,0) and z® = (3, 0) 
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6-1 (a) wi = $, w3 = 2 (b) Aw = (2, 3) @ check 
constraints strictly satisfied 

(©) Amax = 77 (f) wW = (77, 77) 

6-3 (a) no (c) no (e) yes 

6-4 (a) no (c) yes (e) no 

6-5 (a) 2 Aw; + 3 Aw, —3 Aw; = 0, 

4 Aw, —1Aw2+1Aw3=0 


6-6 (a) Ax = (—0.8, —1.6, 4), 


i, Zeal 
Ax = 
(ts 1 a iy 


6-7 (a) d = (—14, —3, —5) 


(b) P = 


—1 
o(? 0 yaa 


(14, 3,5) - (—4,,-7) <0 

6-9 (a) (5, 4,1, 3) © G11, 3) 

6-10 (a) (2,5, 1, 9) (©) (6, 25, 1, 54) 

6-11 (a) x* = (0, 2,0) @) y® = (1,1, 0), 

y* = (0, 2, 0) (c) min 4y, + 3y2 + 5y3, s.t. 

4y + 5y2 + 3y3 = 12, yi, yo, y3 2 0 

6-13 (a) Ax = (—0.64, 1.6, —2.24), 

Ay = (—0.32, 1.6, —2.24) (b) (2, 3,5) - Ax <0, 
(2,5,3) - Ax =0 (c) A = 0.36084 

(d) x = (1.769, 1.577, 0.192), 

y = (0.884, 1.577, 0.192) 

6-15 (a) check feasible and strictly positive (b) min 
40y, + 3y>, s.t. 4y, _— 3y2 + 2y3 = 3, 3y2 —3= 2, 
V1, 2, ¥3 = 0 (¢) Ax = (—7.669, 7.669, 7.669) 


WIE AIR aI 
WIR AIP ale 
WIN WI wie 


0 iL l 
(e) A = 0.12037, x = (3.077, 3.923, 1.923) 
6-17 (a) yes, unbounded (c) no, 
x2) — (3,949, 0.514, 9) 
6-18 (a) w* = (4,0, 0) (b) max 


9 00,1,0)« ax <0,( i ie : )ax=a 


13, = 2w +w3+ w(in(wy) ae In(w2) + In(w3)), s.t. 


3w, + 6w2 + 4w3 = 12, wy, w2, w3 > 0 (ce) 17.7 and 
16.45, 3.088 and —78.10, moderate bonus in middle 
vs. major penalty near boundary 
(d) w* = (3.850, 0.035, 0.060), 
w* = (2.799, 0.285, 0.474), 
w* = (3.850, 0.350, 0.600) 
6-20 (a) yes (¢) no 
6-21 (a) curve II 
6-22 (a) check feasible and strictly positive (b) min 
4x1 — x2 + 2x3 — 10[In(x1) + In(x2) + In(x3)], st. 
4x, — 3x9 + 2x3 = 13, 3x2 — x3 = 1, x1, x2, x3 > O 
(c) Ax = (—4.6415, 6.1887, 18.566) 
(d) (0.667, —11, —3) - Ax <0, 

; 4 . #e Ax = 0 (©) Amax = 0.64634, 
A = 0.10 (f) decrease, then increase 
6-24 (a) check feasible and strictly positive 
(b) Ax = (—16.899, 16.899, 16.899) 
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(c) (7.5, —2.333, -10) - Ax <0, 
1 -1 2 
0 1 -1 
(d) Amax = 0.2367 (e) A = 0.1, 
x = (2.310, 4.690, 2.690) 


Ax =0 
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7-1 (b) dollars of profit, thousand beta zappers, 
thousand freeze phasers, extrusion hours, trimming 
hours, assembly hours (ce) make thousand beta 
zappers, make thousand freeze phasers (d) input1: 
5 hours extrusion, 1 hour trimming, 12 hours 
assembly; outputl: thousand beta zappers, $2500 
profit; input2: 9 hours extrusion, 2 hours trimming, 
15 hours assembly; output2: thousand freeze 
phasers, $1600 profit 

7-3 (b) $ cost, professional-equivalent hours 
production, Proof hours supervision, grad 
maximum hours (c) ugrad hour programming, grad 
hour programming, professional hour programming 
(d) inputl: 0.2 hour Proof supervision, $4 cost; 
output: 0.2 professional-equivalent hour 
programming; input2: 0.15 hour Proof supervision, 1 
hour grad maximum, $10 cost; output2: 0.3 
professional-equivalent hour programming; input3: 
0.15 hour Proof supervision, $25 cost; output3: 1 
professional-equivalent hour programming 

7-5 (a) tighten, decrease, more steep (c) relax, 
decrease, less steep (e) relax, increase, less steep 
(g) tighten, increase, more steep 

7-6 (a) tighten, decrease (c) tighten, increase 

7-7 (a) increase, more steep (c) increase, less steep 
7-8 (a) v; 4 dollar change in optimal profit per 
thousand increase in beta zapper demand, v2 4 
dollar change in optimal profit per thousand 
increase in freeze phaser demand, v3 4 dollar 
change in optimal profit per hour increase in 
extrusion capacity, vs 4 dollar change in optimal 
profit per hour increase in trimming capacity, vs 4 
dollar change in optimal profit per hour increase in 
assembly capacity (b) v1, v2 < 0; v3, v4, vs = 0; RHS 
increase tightens > and relaxes < 

(c) vy + 5v3 + vg + 1205 > 2500, 

v2 + 9v3 +24 + 15vs5 > 1600, implicit cost of 
activities should equal or exceed objective function 
return (d) min 10v,; + 15v2 + 320v3 + 300v4 + 4800s, 
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minimize total implicit cost (f) x; = 10 or v; = 0, 
x2 = 15 or v2 = 0, 5x; + 9x2 = 320 or v3 = 0, 

x; + 2x. = 300 or v4 = 0, 12x, + 15x. = 480 or 

vs = 0, either a primal constraint is active or small 
RHS change has no objective function impact 

(g) v, + 5u3 + v4 + 12s = 2500 or x; = 0, 

v2 + 903 + 2v4 + 15v5 = 1600 or x2 = 0, a primal 
variable should be used only if its implicit cost 
matches its objective function return 

7-10 (a) v; 4 dollar change in optimal cost per 
professional-equivalent hour increase in required 
production, v2 4 dollar change in optimal cost per 
hour increase in Proof supervision, v3 4 dollar 
change in optimal cost per hour increase in grad 
availability (b) v; > 0; v2, v3 < 0; RHS increase 
tightens > and relaxes < (¢c) 0.2v; + 0.2u2 < 4, 
0.3y,+ 0.152 + v3 < 10, v4) + 0.15v2 < 25, implicit 
value of activities should not exceed objective 
function cost (d) max 10001; + 164v2 + 500v3, 
maximize total implicit value 

(f) 0.2x, + 0.3x2 + x3 = 1000 or v; = 0, 

0.2x, + 0.15x2 + 0.15x3 = 164 or v2 = 0, x2 = 500 or 
v3 = 0, either a primal constraint is active or small 
RHS change has no objective function impact 

(g) 0.20; + 0.2v2 = 4 or x; = 0, 

0.3¥,;+ 0.15v2 + v3 = 10 or x2 = 0, v; +0.15v2 = 25 
or x3 = 0, a primal variable should be used only if its 
implicit value matches its objective function cost 


7-12 (a) max 40v; + 10v2, s.t. 2u; + 402 < 17, 

3v, + 4u2 < 29, 2v, + 303 < 0, 3; + v2 — v3 < 1, 

vy < 0, v2 => 0, v3 URS (c) min 10v; +192 + 5v3, 
s.t. 20; + v3 > 30, —3v, + 4v2 + v3 = 0, 

=U) + U3 = —2, ou, =10,v, > 0,m < 0, V3 URS 
(e) min 10v; + 1103, s.t. 3v; + vo = 0, 2v; + v3 > 2, 
—v, + 3v3 > 9, —v2 + v3 = 0, vy < 0, v2 => 0, v3 URS 
(g) max 40v; + 18v2 + 1103, s.t. 15v; + 12. < 0, 
15v; — 90v2 < 32, 15, < 50, 1402 + v3 < 0, 

vz, < —19, v; URS, v2 = 0, v3 < 0 

7-13 (a) 2x + 3x2 + 2x3 + 3x4 = 40 or y= 0, 

4x, + 4x2 + x4 = 10 or v2 = 0, 20; + 402 = 17 or 

x, = 0, 3v,; + 4v2 = 29 or x2 = 0, 20; + 3v2 = 0 or 
x3 = 0, 3v, + v2 — v3 = 1orx =0 

(c) 2x, — 3x2 + 9x4 = 10 or vy; = 0, 4x. — x3 = 19 or 
v2 = 0, 2v; + v3 = 30 or x; = 0, —v2 + v3 = —2 or 
x3 = 0 (e) 3w + 2x; — x2 = 10 0rv,; = 0, w- y=0 
or v2 = 0, 2v; + v3 = 2 or x, = 0, —v, + 303 = 9 or 


x2 = 0 (g) 12x — 90x2 + 1424 = 18 or v2 = 0, 

x4 = 11 or v3 = 0, 15v; +12v2 = O or x; = 0, 

15v, = 90v2 = 32 or xy2= 0, 15v, = 50 or 3= 0, 
1402 + v3 = 0 or x4 = 0 vy = —19 or x5 = 0 

7-14 (a) min 14v; + 14v2, s.t. 2v; + 5v2 > 14, 

Sv} + 2v2 = Us V4, 02 = 0; x (2, 2), v= G, ) 

(c) max 24v; + 11v2, s.t. 20; + 3u2 < 8, 

Ov, tv < 11, v1, v2 = 0; x* = (3,2), v* = C1, 2) 
7-15 (a) vy, =3, 2 = ; (c) vy) = x =4 

7-16 (a) min 4v,; + 122, s.t. 20; = 4, 01 + 3u2 > 1, 
v1, < 0, v2 > 0; unbounded, infeasible (c) max 

2v; + Suz, s.t. vy > 10, vy — vo > 3, 4, v2 > 0, 
infeasible, unbounded (e) max 2v; + 5vz, s.t. 

—vy +2 < —3,2v; — 2v2 < 4, v1, v2 > 0; infeasible, 
infeasible 

7-17 (a) dual optimal value < 70 

7-19 (a) no; 51.25 (b) $0; $208.33 (c) increase 
$20,833; increase at least $39,375 and at most 
$41,667 (d) increase $31,875; decrease at least 
$25,925 and at most $31,875 (e) 87.5 hours 

(£) $624.99 

7-21 (a) v; = $25.882 (b) $1294.10, at least $2588.20 
(c) yes, increase $82.35; increase at least $376.45 
(@) $3.118 

(e) $4917.65; at least $17,376.01 and at most $18,670.58 
(f) yes (g) no (h) $20.12 
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8-1 (b) (166,85.2) to (0,184.8) (ce) first: 

x* = (24, 166, 50); second: x* = (24, 0, 216) 

8-3 (b) (2.4,19.8) to (2.8,12.5) to (3.2,12.1) to 
(4.2,11.3) (c) first: x* = (0, 1, 0, 0,1); second: 

Xe — (lO lee) 

8-4 (b) approximately (9.05,4202) to (12,2760) to 
(20,1809) to (30,1365) to (43.29,1110) (c) first: 

x* = (6.65, 5.56, 4.46, 3.33); second: 

5 Geen I LSI) 

8-5 (a) first: x* = (0, 6), second: x* = (4, 1) (b) no, 
no, yes, yes, no, yes (c) (9,4) to (27,2) to (30,0) 

8-7 (a) first: first objective, s.t. original constraints, 
points (1, 3) through (3, 3) alternative optima; 
second: second objective, s.t. original constraints 
and 6x; + 4x. > 26, x* = (1, 2) (@b) first: second 
objective, s.t. original constraints, x* = (0, 5); 


second: first objective, s.t. original constraints and 
x, > 5,x* = (0,5) 

8-9 (a) min 15x; + 110x2 + 92x3 + 123x4 

8-10 (a) x* = (1, 3) () x* = (0,5) 

8-12 (a) min d; + ds, s.t. 3x, + 5x2 — x3 — dy < 20, 
11 x2 + 23x3 + dy > 100, d;, d> > 0, and all original 
constraints (ec) min d; + d> + d3, st. 

40x; + 23x2 +d, > 700, 20x; — 20x. — d> < 25, 

3x2 + x3 — d3 < 65, d;, do, d; > 0, and all original 
constraints (e) min d; + dp, s.t. 

22x, + 8x2 + 13x3 — d, < 20, 

3x1 + 6x2 + 4x3 + do > 12, d,, d, > 0, and all 
original constraints 

8-13 (b) min d, + d, st. x1 + d, > 3, 

2x1 + 2x9 + dp > 14, 2x) +.x2 <9, x1 <4,m <7, 
X1,X2, d,, d2 > 0 (c) least total distance to the two 
contours 

8-15 (a) min dj, s.t. x, +d; > 3,2x; +2x +d) > 14, 
2x1 +x. <9, x1 <4, x2 <7, x1, x2, di, dy > 0; any 
feasible solution with x, > 3 is alternative optimal; 
min d, s.t. d; < 0 and other constraints of the first 
LP; x* = (3,3) (b) min 100d, + do, s.t. same 
constraints as first LP in part (a) (c) yes; yes 

8-17 (a) first: first objective s.t. original constraints, 
x* = (24, 166, 50); second: second objective s.t. 
original constraints and x2 > 166, x* = (24, 166, 50) 
(b) max 0.50x; + 2.20x2 + 0.80x3, s.t. original 
constraints, x* = (24, 166, 50) (ce) min d, + do, s.t. 
X2 +d; > 100, 0.50x, + 0.20x2 + 0.80x3 + d, > 144, 
d,, d, > 0, and all original constraints 

(e) x* = (24, 100, 116), in between the earlier 

(f) first: min dj, s.t. constraints of part (c), any 
feasible solution with x. > 100 is optimal; second: 
min d2, s.t. d, < 0 and constraints of part (c), 

x* = (24, 100, 116) (g) min 10,000d, + a, 

8-19 (a) first: first objective s.t. original constraints, 
x* = (0, 1, 0,0, 1); second: second objective s.t. 
original constraints and 

1.0x; + 0.4%. + 1.43 + 1.8x4 + 2.0x5 < 2.4, 

x* = (0, 10, 0, 0, 1) (b) min 

4.5x; + 6.2x2 + 7.4x3 + 7.8x4 + 18.4x5, s.t. original 
constraints, x* = (1, 0,1, 1, 0) (ce) min d; + dy, s.t. 
1.0x; + 0.4x2 + 1.4%3 + 1.8x4 + 2.0x5 — d, < 3.0, 
2.5x1 + 5 4x2 + 4.6x3 + 4.2x4 + 14.4x5 — dz < 12, 

d,, d, > 0, and all original constraints (d) part (c) 
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optimal value # 0 (e) x* = (1, 1, 0, 1, 0), new 
intermediate (f) first: min dj, s.t. constraints of part 
(c), x* = (1,1, 1, 0, 0); second: min dz, st. d; < 0 
and constraints of part (c), same x* (g) min 
10,000d, + dz 

8-21 (a) x; A singles, x. 4 doubles, x3 4 luxuries; 
min 2d, + dz,s.t. 40x; + 60x. + 120x3 < 10,000; 
0.7x; + 0.4x2 + 0.9x3 + dy > 100; 

0.3x, + 0.6x2 + 0.1x3 + dy > 120; all variables 
nonnegative (b) x} = 76.92, x} = 115.38, x3 = 0.00, 
d,* = 0.00, d;* = 27.69 

8-22 (b) nonzeros: xj = 4.4, x} = 10.0, x¥ = 12.3, 
x4 = 16.5, rf = 0.6, rj = 1.5 

8-28 (b) v* = 285, f* = 0.007, d* = 0.040, 

81° = 8;" = 0,83 = 0.637 

8-32 (¢) x} = 53.33, x3 = 500, x} = 166.67, 

yj = 100, y; = 1150, y3 = 250, all dt = 0 except 

di = 466.67 


Chapter 9 


9-1 (a) nodes: 1,2,3,4,5; arcs: (3,2), (3,4), (3,5); 
edges: (1,2), (1,3), (1,4), (2,5), (4,5) (b) yes, no, yes, 
no (c) replace each edge with two opposed arcs of 
the same length as the edge 

9-3 (a) 1-3-5-2, length 6, 1-3, length 3, 1-4, length 
7, 1-3-5, length 4 (b) 1-3 best for 1 to 3, 1-3-5 best 
for 1 to 5, 3-5 best for 3 to 5, 3-5-2 best for 3 to 2, 
5-2 best for 5 to 2 (c) v[1] = 0, v[2] = 6, 

x1,3[2] = X3,5[2] = X2,5[2] = il v[3] = 3, x13[3] = by 
v[4] = 7, x1,4[4] = 1, v[5] = 4, x13[5] = x3,5[5] = 1 
(@) v[1] = 0, v[2] = min{v[1] + 10, v[3] +5, v[5] + 2}, 
v[3] = min{v[1] + 3}, 

v[4] = min{v[1] + 7, v[3] + 9, v[5] + 4}, 

v[5] = min{v[2] + 2, v[3] +1, v[4] + 4} (f) positive 
lengths preclude negative dicycles 

9-5 (a) 1-3-2, length 4, 1-3, length 3, 1-3-2, 
length 10, 2-1, length 8, 2-4-3, length 10, 2-4, 
length 6, 3-1, length 3, 3-2, length 1, 3-2-4, length 
7, 4-3-1, length 7, 4-3-2, length 5, 4-3 length 3 

(b) 1-3 best for 1 to 3, 1-3-2 best for 1 to 2, 3-2 best 
for 3 to 2, 3-2-4 best for 3 to 4, 2-4 best for 2 to 4 
(c) v[k][k] = 0,k =1,...,4, v[1][2] = 4, 

xia[1[2] = xs2[1]2] = 1, vft[3] = 3, 3[1]13] = 1, 
vf1][4] = 10, x1 3f1]14] = x21 1]l4] = x2 s[1]4] = 1, 
21] = 8, xsal2IC1] = 1, vf2][3] = 10, 
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%2,4{2][3] = x4,3[2][3] = 1, v[2][4] = 6, x2,4[2][4] = 1, 
v[3][1] = 3, x1,3[3][1] = 1, v[3][2] = 1, x3,2[3][2] = 1, 
v[3][4] = 7, x3,2[3][4] = x2,4[3][4] = 1, 

v[4][1] = 7, x43[4][1] = x13[4][1] = 1, 

v[4][2] = 5, x4,3[4][2] = x3,2[4][2] = 1, v[4][3] = 3, 


= min{8, v[1][3] + v[3][2], Jf} 
v[1][3] = min{3, v[1][2] + v[2][3], v[11[4] + v[4] (2), 
v[1][4] = min{v[1][2] + v[2][4]. v[1J[3] + vBI[4h, 


If 

Il 

IE 

Il 
v[2][4  Y[2][1] + vf1][4], »[2][3] + [3][4]}, 
v[3][1] = min{3, v[3][2] + v[2]f1], v{3][4] + vf4][1}, 
v[3}[2] = min{1, v[3][1] + v[1][2], v[3][4] + vi4][2]}. 
v[3][4] = min{v[3][1] + v[1][4]. »[3][2] + v[2][4}} 
y[4][1] = min{y[4][2] + v[2][1], v[4][3] + v[3]]}, 
v[4][2] = min{6, v[4][1] + v[1][2], v[4][3] + v[3][2]}, 
v{4][3] = min{4, v[4][1] + v[1][3], v[4][2] + v[2][3}} 
(f) positive lengths preclude negative dicycles 


9-7 (a) 1-2, length -5, 1-3, length 10, 1-3-4, length 
12 (b) 1-2-4-1, 1-3-4-1 (ce) no, yes (d) functional 
equations nolonger sufficient 

9-9 (a) one to all, no negative dicycles (b) see 
Exercise 9-3(a) (c) see Exercise 9-3(a) (d) 1-3-2, 
length 8, 1-3, length 3, 1-4, length 7, 1-3-5, length 4 
(e) 4, labels correct after t = 4 

9-11 (a) 1-3-4-1 

9-12 (a) all to all, no negative dicycles (b) see 
Exercise 9-5(a) (c) see Exercise 9-5(a) (d) 1-2, 
length 8, 1-3, length 3, 1-2-4, length 14, 2-1, length 
8, 2-1-3, length 11, 2-4, length 6, 3-1, length 3, 3-2, 
length 1, 3-2-4, length 7, 4-2-1, length 14, 4-2, 
length 6, 4-3 length 3 

9-14 (a) 1-3-4-1 

9-15 (a) one to all, lengths nonnegative (b) see 
Exercise 9-3(a) (c) see Exercise 9-3(a) (d) 1-3-2, 
length 8, 1-3, length 3, 1-4, length 7, 1-3-5, 

length 4 

9-17 (a) Bellman—Ford, Floyd—Warshall, Dijkstra 
(best), 1-3-2, length 5, 1-3, length 2, no path to 4 
(c) none apply 


9-18 (a) acyclic; one numbering: a = 1, b = 3, 
c=4,d=2,e=6,f =5 (ce) not acyclic; dicycle 
a—d-e-c—b-a 

9-19 (a) acyclic digraph (b) v[1] = 0, v[2] = ov, 
v[3] = 2, v[4] = 12, v[5] = —1, v[6] = 11 (©) 133, 
1-3-4, 1-3-5, 1-3-5-6 

9-22 (a) 


(b) check arcs (i, j) have i <j (e) v[1] = 0, v[2] = 0, 
v[3] = 5, v[4] = 1, v[5] = 2, v[6] = 2, v[7] = 0, 

v[8] = 4, finish = 8 () start-1-3-finish (e) late 
starts: 1 =2,2=6,3=7,4=7,5=8,6=9, 
P=378 7: slacks; i= 52 —O8s:— 21 4- =16, 
S656 157-3: i= 3 


9-24 (a) 


(b) one numbering: F = 1, CS = 2, 1LB = 3, 1IW = 
4,2FL = 5, 1FN = 6, 2LB = 7, 2IW = 8, R = 9, 2FN 
= 10 (e) v[F] = 0, v[CS] = 8, v[1LB] = 13, 

v[1IW] = 13, v[2FL] = 16, v[1 FN] = 19, 

v[2LB] = 19, v[27W] = 19, v[R] = 23, v[2FN] = 25, 
finish = 35 

(d) start-F-CS-1LB-2FL-2LB-R-2FN-finish 

(e) late starts: F = 0, CS = 8, 1LB = 13, 1[W = 19, 
2FL = 16, 1FN = 23, 2LB = 19, 2IW = 20, R = 23, 
2FN = 25; slacks: F = 0, CS = 0, 1LB = 0, 1I1W = 6, 
2FL = 0, 1FN = 4, 2LB = 0, 2TW = 1, R = 0. 2FN 
=0 


9-26 (b) 


9-28 (b) 


(d) take 1 and 3 

9-30 (c) for 8: 1-2-13-3-8, length 26; for 10: 
1-2-13-3-8-10, length 41; for 11: 1-2-13-4-11, 
length 19; for 12: 1-2-13-3-8-6-12, length 35 

9-31 (c) 1-2, length 6; 1-2-7-3, length 42; 1-2-7-4, 
length 30; 1-2-7-5, length 20; 1-6, length 11; 1-2-7, 
length 12; 1-2-7-5-8, length 34; 1-2-7-5-8-9, 
length 57; 1-2—7-5-8-10, length 59 

9-32 (b) yes (d) 7—11-2 with one 4-hour and one 
3-hour, cost $90 

9-33 (d) 1-3-6-7 (or 1-3-5-7), length 9 gridsize 
9-34 (d) using S for shed and I for the unlabeled 
intersection: S—S—I-1, length 24; S-5—3—2, length 26; 
S-5-3, length 14; S-5-4, length 25; S-5, length 5 


Chapter 10 


10-1 (a) V = {1, 2, 3, 4, 5}, 
A={(1,2), 1,4), 2,5), G6, D,G.4), 
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(3,5), (4, 2), (4,5), (5, 2)} (b) source: 1,3; sink: 2,5; 
transshipment: 4 (d) min 5x, 2 + 10x14 
—6x3,1 + 2x34 + 8x42 + 6X45 + 1x5,2 + 3x53, 


Soi 4 a S50; 
X12 — X2,5 +X4,2 +.X5,2 = 20, 
—X3,1 — X3,4 +X53 = —70, 


X14 +%3,4 — X42 — X45 = 0, 
X25 + X4,5 — X5,2 — X53 = 100, 
X14 40, x2,5 < 10, Xa 4, X42 < 20, all Xj,j => 0 


(e) A= 
-1 -1 ik 0 0 0 0 0 
1 0 -1 0 il 0 1 0 
0 0 —1' -l 0 0 0 1 
0 I 0 0 1 -1 -1 0 0 
0 0 1 0 0 0 1 -1 -1 


10-3 (a) check every column has at most a —1 anda 


+1) 
= (2) 
@) @) 


10-5 (a) 150 > 80 (b) node 4, demand 70; zero cost 
arcs (1,4) and (2,4) 


10-7 (a) min 25xp1,.c1 + 30Xp1,c2 + 15xpi.w + 
45xp2,c1 + 23Xp2,c2 + 15xp2,w + L1xw.c1 + 14x9,c2, 
St. Xpi.c1 +Xpi,c2 + Xpiw < 400, 

Xp2.c1 +Xp2,c2 +Xp2,w < 600, 

Xpiw +Xpo,w — Xw.c1 —Xw.c2 = 0, 

Xpi.c1 + Xp2,c1 + Xw.c1 = 160, 

Xpi.c2 + Xp2,c2 + Xw,c2 = 700, all x;; = 0 

(b) X51 ¢ = 160, x%) y = 100, x$) oy = 600, 

XW.co = 100 


(c) 


(d) source: P1,P2; sink: C1,C2,D; transshipment: W 
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10-9 (a) cycle (b) chain, path (ce) chain (d) cycle, 
dicycle 

10-11 (a) check flow balance and capacities 

(b) Ax = 

+ (1, 0,1, —1, 0), +(0, 1, 0,1, —1), +, 1, 1, 0, —1) 
(d) no, yes, yes, no, no, no (e) no, yes, yes, no, no, yes 
(f) 20, 20, 10 

10-13 (a) x* = (0, 35, 60, 40, 0) 

10-15 (a) add node 0; (1,0) flow 50, (3,0) flow 70, 
(0,2) flow 20, (0,5) flow 100 

10-16 (a) yes (c) no 

10-17 (a) min 

23Xxn,B + TIXN.W + 8x5 2 + 94x Ss w + 53xrB + 41xrw, 
S.t. Xv B+ XNW = 50, Xs p+Xsw = 50, 

Xrp+Xrw = 50, xy,.p +Xs.8 +X7.B = 60, 

XN,W +Xswtirrw = 90, all Xi,j = 0 (b) XN RB — 10, 
XNw = 40, x§ 3p = 50, x7 y = 50 


(c) 
(d) 
Bo oW 
23] 177 
N| 10] 40] 50 
8] foal | 
S| 50/0} 50 
8] 94 
1 0/50] 50 
60 90 


10-19 (a) max Siam ae rjjXij, Sut. ei ie 
i=1,...,4, 2.9 =1j=1,...,4,all 

x;; = 0 or 1, where r,; & rating of member i for task 
JD) x}, = X44 = X33 =X}. = 1 (©) 4 member 
nodes with supply 1, 4 task nodes with demand 1, 
arcs from every member to every task with cost the 


negative of the corresponding rating (d) network 
flow with integer supplies and demands 


10-21 (a) supply 1 at 3, demand 1 at 2, all other net 
demands 0, x* = (0, 0, 0, 1, 0) (c) supply 1 at 1, 
demand 1 at 4, all other net demands 0, 

x* = (1,0,1,0,1) 


10-23 (a) 


470 


(b) <4, 3 = 400, x ¢ = 470, x} p = 80, x} - = 320, 
Xé p = 470, xp ~ = 550 (c) add infinite capacity, cost 
= —larc from E to A; all other cost = 0; all net 
demands = 0 


10-25 (a) add infinite capacity, cost -1 return arc 
(2,3), all other costs and all net demands 0; 

x* = (30, 0, 30, 100, 0) (¢) add infinite capacity, cost 
-1 return arc (4,1), all other costs and all net 
demands 0; x* = (50, 40, 0, 50, 40) 


10-26 (a) min 25xp11,c1,1 + 30XP11,c2,1 + 

15xpia,wa + 45xp2,1,c1,1 + 23%P2,1,02,1 + 15XP2,1,Wwa + 
L1xwi.cia + 14%¢W1,c21 + 25%p1,2,¢1,2 + 30%P1,2,c2,2 + 
15xp12,w2 + 45Xp2,2,c1,2 + 23Xp2,2,c2,2 + 15xXp2,2,w,2 + 
11xw2,c1,2 + 14% W.2,c2,2 + 10xwi,we, st. 

+ Xpii.wa < 400, 

+ Xp2,1,w.1 < 600, 


XP11,C1,1 + XP1,1,c2, 
XP2,1,C1,1 + ¥P2,1,€2, 


XP11,w +Xp2,1,w.1 —Xw.1,c1,1 —Xw,1,c2,1 —Xw,w2 = 9, 


XP11,C11 + Xp21,c1,1 + Xwi,cia = 150, 
XP11,02,1 + Xp2,1,02,1 + Xwa,c2,1 = 700, 


Xp1.2,c1,2 + XP1,2,02,2 + Xpi2,w2 < 400, 
Xp2,2,C1,2 + ¥p2,2,€2.2 + XP2,2,w,2 < 600, 


XP1,2,W.2 +Xp2,2,w,2 —Xw.2,C1,2 —Xw,2,02,2 + Xw.1.w2 = 9, 
Xp1,2,c1,2 + Xp2,2,c1,2 + Xw2,c1,2 = 300, 

Xp1.2,€2,2 + Xp2,2,c2,2 + Xw,2,c2,2 = 810, all x,4 5; = 0 
(D) Xh1 4,011 = 160, X14 1 = 210, X11 = 600, 
XW1,c2.1 = 100, Xp1.2.c1,2 = 300, xp, 2 w2 = 100, 


Xp2.2,¢2,2 = 600, Xiy-2,c2,2 = 210, Ywiw2 = 10 


(c) 


10-28 (a) apply weights +1, +1 (c) apply weights 
+1, —-1, -1 

10-29 (a) 13 = 50, 14> 10, X23 = 25, X25 =5, 
X3,5 = 65, x45 = 10; feasible (c) x1,2 = 25, x14 = 35, 
X23 = 55, X35 = 45, x45 = 35; infeasible (e) not 
basis, cycle 1-3-2-1 (g) not basis, not connected 
10-30 (a) forms spanning tree, all off-tree at bounds 
(b) increase (2,3), Ax = (1, —1, 1, 0, 0); increase 
(3,4), Ax = (-1, 1, 0, —1, 1) (€) yes,yes (d) 15,10 
10-32 (a) x3, = 35, x3, = 60, x3, = 40 (©) x7, = 15, 
35, x33 = 15, x3, = 10 


X13 
10-33 (a) check flow balance and capacities 


(b) 


(c) direction of 1-2-3-1 or 2-3-4-2 (d) 15 or 10 
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10-35 (a) x3, = 35, x3, = 60, x3, = 40 (©) x}, = 15, 
xf = 35, x53 = 15,23, = 10 


10-36 (a) feasible, spanning tree, all off-tree at 
bounds (b) 2-3-4-2 (c) 4-2-1-3-+4 


9 a 
10-37 (a) min 0) %K,1,3 + Xk2,1 + Xe3,2), 
S.t. X%1,3.1 + X1,2,1 


X1,1,2 — 11,3 = 7; 
X1,1,2 + X1,3,2 — X1,2,3 — X1,2,1 = 0, 
His te Me ge aoe — "7, 


X2,3.a + X9,2,1 — X94,0 —\X04,3 = 7, 


X10: + X930'— X03 =o —7, 
2,23 + X2,1,3 — X2,3,1 — X2,3,2 = 0, 
x 43.91 — 3342 — 3313 = 0) 
X3,1,.2 +332 — %3,23 —%32,1 = 7, 
X32,3 + %3,13 — 43,31 —%332 = —7, 


Yeates S11 Ses a 
ent Xk3.1 < 11, all xe 5; = 0 ) 


* ae: _— * a * — * —_ a aw 
X12 =%312 =X 23 = 23 =%331 = 331 = 95, 
* aK ee ak 

113 =%321 = 332 =15 


(0, -7, 7) 


(d) an optimal solution ships each 7 units to itself at 
zero cost (e) integer flows not guaranteed in 
multicommodity flows 


10-39 (a) min 7(x1,1 + 1,2) + 82,1 + X2,2) 
+5(03,1 +32), St. 7, iy < 300,8=1,...,3, 
0.7x1,1 + 0.8x2,1 + 0.6%x3,1 = 350, 

0.6x1,2 + 0.8x2,2 + 0.7x3,2 = 275, all x,; = 0 

(b) x}, = 300, x3, = 175, x3. = 81.25, x3, = 300 


898 


Selected Answers 


(c) 


(d) integer flows not guaranteed in flow with losses 


Chapter 11 


11-1 (a) proportional to variable magnitude 

(b) x* = (96.67, 5, 48.33) (c) min 

2250y1 + 1650y2 + 2700y3, s.t. 

150y, + 150y2 + 150y3 = 150, 

300y1 + 600y2 + 300y3 < 310, 

600y; + 450y2 + 150y3 < 450, y; = 0 or 1, 
j=1,...,3@x* = (0,0, 150), y* = (0, 0, 1) 

(e) min 15x; + 11x. + 18x3 + 400(1 + y2 + y3), St. 
x; < 150y;, yj =O or1,j =1,...,3, and all original 
constraints (f) x* = (0, 0, 150), y* = (0, 0, 1) 

(g) add variables y,, y2, y3, and constraints 
50y; < xj < 150y;, y; =Oorl,j=1,...,3 

(h) x* = (100, 0, 50), y* = (1, 0, 1) 

11-2 (h) x* = (175, 0, 125), y* = (1, 0,1) 

11-3 (a) max 4.5x; + 4.1x2 + 8x3 + 7x4, s.t. 

4x; + 3.8x2 + 6x3 + 7.2x4 < 8x; =Oorl, 
j=1,...,4@)x* = (1, 1,0, 0) 

11-5 (a) budget: $100 total; mutual exclusiveness: 
alternatives for NW and SE parcels; dependency: 
TE tunnel upon IE lab (b) max 

9x, + 2x2 + 10x3 + 2x4 + Sxs + 8x6 + 10x7 + 1g, 
st. 48x; + 20.8x2 + 32x3 + 28x4 + 44x5 + 17.2x%65 + 
36.8x7 + 1.2xg < 100, x, +x4 + x5 + x6 <1, 

X_g +x%7 <1, xg <%7,m1,...,X3 =Oorl 

(c) x* = (0, 0, 1, 0, 0, 1, 1, 1) 


11-7 (a) max 45x; + 30x2 + 84x3 + 73x4 + 80xs5 
+70X5 + 61x7 + 91xg, St. xy +2 < 1,42 +45 <1, 
XgtxX7 <1,x5 +x +4, <1,%1,...,x8 =Oorl 
(b) x* = (1, 0, 1, 1, 0, 0, 0, 1) 

11-9 (a) min 40x; + 65x + 43x3 + 48x4 

+72x5 + 36x65, St. X, +X. > 1.x, +x4> 1, 

X2 +%3 > 1, x2 4+%5 > 1,%3+x5 > 1,x3+%5 > 1, 
xg 4X5 >1,x5+%6 > 1,%,...,xX%5 =Oorl ) 

x* = 01,0 1,0; 1) © min }o yast 

yy +x%+y, > 1, 

X+%4+y2 > 1,%2+4%3+y3 > 1, 

X2 +X4 + y4 > 1,%2 +%5+ys5 > 1, 

X3+%5+ yo > 1, x3 +%6+y7 = 1, 

X4+X5 + yg >1,x5+%6 + yo = 1, 

ee 2D) AG, ois KOO Or Le 

y1,---, ¥9 =Oor1 @) x* = (0,1, 0, 0, 1, 0), 

y* = (0, 1, 0, 0, 0, 0, 1, 0, 0) 

11-11 (a) min 1.40x, + 0.96x2 + 1.52x3 + 1.60x4 + 
1.32x5 + 1.12x6 + 0.84x7 + 1.54x¢, 

S.t. Xo +X%5 +xg = 1,41. +43 4x4 =1, 

Xs tXxXotx%g=1,%, +%3 +47 +x, = 1, 

X3 + X4 + xX6 = 1, x2 +24 +245 +x, =1, 

Mis eotioX ey = Olotel 

(b) x* = (0, 0, 1, 0, 1, 0, 0, 0) 

11-13 (a) min 18x11 + 26x12 + 31x14 + 50x22 
+22x,3 + 40x31 + 29x32 + 52x33 + 39x34 + 43x43 + 
46x44, S.t. X11 +442 +%14 = 1, x22 +223 =1, 

X31 + X32 + X33 +%3,4 = 1, %43 +%44 = 1, 

X41 +%31 = 1, x12 +%22 + X32 = 1, 

X23 +33 +%4,3 = 1,%144+%34+%44 = 1, 

all x;,; > 0 (b) can be viewed as a network flow with 
unit supplies and demands 

(C) x}, =%33 =X}. =Xj4=1 

11-15 (a) min 60x41% 42 + 120xq1xE3 + 
36xH1XM.2 + 72XH1X%M3 + 60X 4 2X E14 + 20x72 F,3 + 
36XH2%uMa1 + 12xH2Xm3 + 120XH3Xp4 + 20% 173% 8,2 ar 
72X43XM,1 + 12X43XM,2 + 42xXp1%mM2 + 84x e1%M3 + 
42x p2xXm1 + 14x¢2xm3 + 84x 2 3%m1 + 14x53xm2, 
S.t. X41 +X%H2 + %H3 = 1, X21 +%22+%03 = 1, 
Xuma t+%m2+%4u3 =1,x41+%e1 +xXmi =1, 
Xu2+XE2 +%mM2=1,%03+%53+xm3 = 1, all 
X;; = 0 or 1 (b) can assess cost only after pairs of 
assignments (¢) 773 =Xh. =Xy, = 1 

11-17 (a) min ©, (cirxir + ¢.px;5), St. 

hip + ¥ip = 1a = 1,-1.6, ee = 200, 


ES, Xz < 190, all x;; = 0 or 1, where c;; 4 
materials handling to move i to j and ¢; A time fori 
(b) more than one unit of time allocated by each 
decision (c) x7 p 
X33 =Xop =1 


11-19 (a) min YP) igs Cai, St 


a oo * 
X7 p= %3 7 = Xa er 


X12 $413 +X 4+%15+%X16 = 1, 
X12 +23 + X24 + X2,5 + x26 =1, 
X43 + X23 + 23,4 + 3,5 + 3,6 = 1, 


X14 + X24 +%3,4 + X45 + X46 = 1, 

X15 +%2,5 +35 + X45 + X56 =1, 

X16 +26 +436 + X46 + X56 = 1, 

all x;; = 0 or 1, where c;; 4 cost of pairing i with j 
(b) objects to be paired do not come from distinct 
sets (€) Xf 5 =X}3 =%}6 =1 


11-21 (a) seeks a minimum total length closed 
route visiting every point 


(b) 
1 2 3 4 5 6 

at — 40 230 160 220 40 
2 40 — 190 120 180 20 
3° 6-230) 190 — 70 30 190 
4 160 120 70 — 60 120 
5 220 180 30 60 — 180 
6 40 20)” SOF 120" 180. = 


(¢) min YP Dyin Cimijs Set 

X42 $%13 +414 +X15+%X16 =2, 

X12 +.X2,3 + X24 + X25 + X26 = 2, 

X13 + X23 +34 +%35 + X36 = 2, 

X14 +X2,4 +34 + X45 + X46 = 2, 

X15 +X2,5 + X35 + X45 + X56 = 2, 

X16 + X26 +%3,6 +X46 +X5,6 = 2, all xi; =0 or 1, 
where d;; 4 i-to-j distance of part (b) 

A) Xt. = Xig = 49.6 = X34 = X35 = X45 = 1 

(e) x13 + 1,4 + X15 + X2,3 + X2,4 

+X2,5 +36 + X46 + X5,6 = 2 

(B) X19 = Xig = 493 = X35 = X45 = X46 = 1 

(g) model with d,; as in part (b) 

11-23 (a) seeks a minimum total setup time closed 
tour of the four products, and times are not 
symmetric (b) min =e igi Cums, Sits isi Derails 
$=1,..54 0% =17=1,....4 allay = 0, 
where c;; 4 given i to j setup time. 

(0) X13 =%31 = 234 =%42 =1 

@) x12 +514 +232 +%34 21 
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(e) xf, = %3,4 = x3, = X43 = 1 (f) model i127) 
with d;; the given setup times 

11-25 (a) min 7 (fiyi + Dey dems), St 
pay Xij Ss 4y;,1= Cast wih Xij= ili 
j=l,....4,y1,-..,y¥3 =O0or 1, all x,; => 0, where 
f, & given fixed cost for i, c;; 4 given transportation 
cost from i to j, and d; £ given demand at j. 

(b) y3 = 23, =%3. =%3, =1 

11-27 (a) min 2X 1,5; + 8000 r1,51 + 2X p27 + 
6000y 42,41 + 2XF2,51 + 10,000 y 72,51 + 2Xr2,52 + 
14,000yr2,52 + 251,52 + 2000ys1,52 + 2X51,7 + 2X52,7, 
S.t. XF2,F1 — XF1,s1 = —800, 


—XF2,F1 — XF2,51 — XF2,52 = —600, 

Xr1,51 + XF2,51 — Xsi,s2 — Xsi,r = 9, 

XF2,52 +Xsi,s2 — Xs2,7 = 0, 

Xsi.t +Xs2.7 = 1400, xF1,51 < 1000yri,51, 

Xp2,F1 < 1000yr2.F1, X¥F2,51 < 1000yr2,s1, 

XF2,52 < 1000y 2,52. Xs1,s2 < 1000y51,52, 

all x;; > 0, all yj =0 or 1 ®) YF 9) =Vin,g = 1, 
XF1,51 = X17 = 800, Xf 59 = X$2,7 = 600 

11-29 (a) x, +10 <x. +M(1—yi2), 

x2 +2 <x, +Myi2, 

x, +10<%x3+ M1 — y13), 

x3 +16 <x, + My13, 

x, +10 <2x4+ Md — yi), 

x4 +8 <x, + My 4, 

X2 +3 <x3+M(1 — yaa), 

x3 +16 <x. + My23, 

X2 +3 <x4+ M1 — yr), 

X4+8 <x2+ Myra, 

x3 +16 <x4+ M(1 — y3,4), 

X4 +8 <x3 4+ My34,%1 > 0, x2 > 20, x3 > 1, 

x4 > 12, all yj = 0 or 1 (b) 39, 23.5, 38, 15.25, 19, 
2.75, 19, 4.75 (c) min 

4% +10 + x2 +3 +x3 +16 + x4 + 8), st. 
constraints of part (a) (d) x} = 0, x3 = 20, x3 = 23, 
xj = 2 (e) mean flow time and mean lateness 

(f) min z, st. Z > x, —2,z7 > x2 — 27,2 >%3—4, 
z > x4 — 13, plus all constraints of part (a) 

(g) xf = 0.x} = 34,x3 = 10, x5 = 26 (h) maximum 
tardiness 

11-31 (a) x11 +10 < x12,%12 +3 < 413, 
X42 S X73, 42,3 PLS X2,25 

X32 +6 < X31,%31 +12 < x33, 

X11 +10 < x21 + Md — yi21), 
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X21 +2 < x11 + My 21, 

X11+10 < x3, + M(1 —yi31), 

x34 +12 < x41 4+ Myi31, 

X21 +2 = x31 + M1 — yo 3,1), 

X31 +12 < x21 + Mya31, 

X12 +10 < x22 + MC — yi22), 

Xo. +2 < X12 + Myi 22, 

X12 +10 < x32 + MQ — yi 32), 

X32 +12 < x12 + Myi 32, 

X22 +2 < x32 + M1 — y232), 

X3,2 +12 < x22 + My232, 

X13 +10 < x23 + M(1 — y123), 

X23 +2 <%134+ Myi 23, 

X13 +10 < x33 + M(1 — yi 3,3), 

X33 +12 < x13 + My, 33, 

X23 +2 < x33 + M(1 — yo33), 

Xeiseets 12 < X23 + My233, all Xk 2 0, all 

Yij,k = 0 or 1 (b) mean flow time (c) min 

$ (413 +14 +222 +4 +233 +8), st. all constraints 
of part (a) @) x7, = 2, x7, = 12, x3, = 15, x3, =0, 
X39 = 6,433 =5,x3, = 12,x3, = 0, x3, = 29 


Chapter 12 


12-1 (a) x* = (0, 1,1, 0) 

12-2 (a) 16, 21 

12-3 (a) yes (c) no 

12-4 (a) same except last two constraints replaced 
by0 <x, <1 

12-5 (a) yes (c) no 

12-6 (a) optimal ILP value < 54.5 (ce) optimal ILP 
value > 19 

12-7 (a) no (c) yes 

12-8 (a) ILP optimal value < 18.5 (b) not optimal; 
¥ = (0, 0, 1, 1, 0) (©) 14 < optimal value < 18.5 

(d) x* = (0, 0, 1, 1, 0), value 14 

12-10 (a) ILP optimal value > 26.5 (b) optimal 
(c) 26.5 = optimal value (d) x* = (0.5, 1, 0, 1, 0.5), 
value 26.5 

12-12 (a) x* = (0, 0, 0, 0), y* = 0 (c) same 
integer-feasible solutions of all = 0 or all = 1 

(d) solution value 0 vs. solution value —42 

12-14 (a) valid; yes (b) valid; no (c) not valid 

(d) valid, yes 

12-16 (a) valid; yes (c) not valid 


12-17 (a) 4, 6 (b) original LP optimum violates 

X2 < 6y2 (c) new xX = (0, 3.2), y = (0, 0.533) 

12-19 (a) max 30x, + 55x2 + 20x3 + v1 (55 — 40x, 4 
12x. — 11x3) + v2(20 — 19x, — 60x2 — 3x3), st. all 
undualized constraints; v; > 0, v2 < 0 

12-20 (a) x7, =x}, = y; = 1 () min 

3X11 + 6x12 + 5x24 + 2x22 } 250y1 t 300y2 + V4 (1 
X11 — X21) + v2(1 — x12 + x22), st. all given 
constraints except the two relaxed (c) optimizations 
are now independent for facilities 1 and 2 (d) all x;; 
and y; = 0, relaxation value 0 < 259 (e) all X;; and j; 
= 0, relaxation value 200 < 259 (f) all ¥,; and J; 

= 1, relaxation value —934 < 259 

12-22 (a) (0, 0, 0), (0, 0, 1), 1, 0, 0), (1, 0, 1) 

12-23 (a) (#, #, #, #), @, #, 1,4), @, 1,1, #, 

(#, 0, 1, #), (#, #, 0, #) (b) branched: 0, 1; 
terminated: 2, 3, 4 (c) 0,3 


12-25 (a) given ILP plus constraints x. = 1, x5 = 0 


12-26 (a) branch on x; = 1 vs. x; = 0 (ce) terminate 
by solving after saving X as a new incumbent 
solution (e) terminate by bound 

12-27 partial solutions: (#, #, #), (#, #, 0), (0, #, 0), 
(0, 1, 0), (0, 0, 0), (1, #, 0), (1, 0, 0), (1, 1, 0), (#, #, 1), 
(1, #, 1), (0, #, 1); x* = (1, 0, 1, 0) 

12-31 0: branch on fractional x3; 1: branch on 
fractional x2; 2: branch on fractional x; 3: 
terminate by solving after saving incumbent 
solution x = (1, 0, 0), > = 90; 4: terminate 
infeasible; 5: terminate by solving after saving 
incumbent solution ¥ = (0, 1, 0), } = 50; 6: 
terminate by bound 54 > 50; x* = (0, 1, 0) 

12-33 (b) 0: round for incumbent solution 

x = (0, 0,1), ) = 54, branch on fractional x3; 1: 
round for new incumbent solution x = (0, 1, 0), 

} = 50, branch on fractional x2; 2: terminate by 
bound 75 > 50; 3: terminate by bound 90 > 50; 4: 
terminate by bound 54 > 50; x* = (0, 1, 0) 

12-35 (a) 18 to 00, 18 to ov, 18 to 00, 18 to 90, 18 to 
90, 18 to 50, 50 to 50 

12-36 (a) a, b; a, b, c (b) 212 (c) 17; 8.7 percent 
12-38 (a) partial solutions: (#, #, #), #, #, 0), 

(#, #, 1), (0, #, 0), (1, #, 0), (1, 0, 0), (1, 1, 0), 4, #, D, 
(0, #, 1); (0, 1, 0), (0, 0, 0); x* = (1, 0, 1, 0) (b) partial 
solutions: (#, #, #), (#, #, 0), (0, #, 0), (0, 1, 0), 


#, #, 1), (1, #, 1), (1, #, 0), (1, 0, 0), (1, 1, 0), (0, #, 1); 
(0, 0, 0); x* = (1, 0, 1, 0) 

12-41 0: fractional so introduce valid inequality 
4x, + 3x2 > 3, which cuts off the current relaxation; 
1: lacking further valid inequalties, branch on 
fractional x,; 2: fractional so introduce valid 
inequality x2 + x3 < 1 which cuts off the current 
relaxation; 3: terminate infeasible; 4: lacking 
further valid inequalities, branch on fractional x3; 5: 
terminate by solving after saving incumbent 
solution x = (0, 1, 1), } = 75; 6: terminate 
infeasible; x* = (0, 1, 1) 

12-43 (a) x* = (1, 1, 0) (b) (0,0,1), (1,0,1), (0,1,1), 
(0,0,2) (ec) x = (1, 0, 1) (@) &X = (1, 1, 0) 

(f) x = (1, 1, 0) 

12-45 (a) x* = (0, 1, 1, 0) (b) X = C1, 0, 0, 0) 

(c) X = (0, 1, 1, 0) 

12-49 x = (0, 1, 1, 0) 

12-51 x = (0, 1, 1, 0) 

12-53 (a) check both feasible (b) after 1 or 2: 

x9) =x) x — x®: after 3: x = (0,0, 1, 1), 

x = (0, 0, 0, 0) (¢) all feasible except x cutting 
after 3; infeasibles must either be excluded from the 
population or included with a large negative 
objective value 

12-55 (0,1, 1,0), value 16; (0, 0, 1, 0), value 9; 

(0, 1, 0, 1), value 15; and any feasible immigrant 
such as (0, 1, 0, 0), value 7 

12-57 (a) x* = (1, 0, 1, 0) (b) most payoff per unit 
constraint usage (¢c) x = (0, 1, 0, 1) 


Chapter 13 

13-1 (a) min 40(x/2) + 2000/(x/5) 

(b) x* = 22.4 

13-2 (b) a* = 0.031 

13-3 (a) min (x1)? + (2 + 30)? 

+y/ (x1 — 50)? + (2 + 10) 

vy (x1 — 70)? + (%2 — 20) 

a/ (1 — 30)? + G2 — 50)? (b) x* = (45.8, 2.7) 
13-4 (b) x* = (27.7, 5.2) 

13-5 (a) min Ss ale — a(u;)? |’, where u; and t; are 
the given units and average time values 

(b) a* = 10.36, b* = —0.322 

13-6 (b) k* = 62.79, a* = 2.011, b* = —0.363 
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13-11 (b) Defining a = am, B = (bm — a), y = —b, 
the fitted equation has the linear form 

a+ Bry. + y(n)? 

13-13 (a) yes (c) no (e) yes 

13-14 (a) local maximum; local and global 
minimum; nothing; local and global maximum; 
nothing; local minimum 

13-15 (a) local maximum; local and global 
maximum; nothing; local and global minimum; 
local minimum 

13-16 (a) no max, x", x; no min, x, x (¢) no 
max, x, x®); no min, x, x (e) yes max; no min 
x =3,-2 

13-17 att =4,x* + 2.47 

13-19 (a) x) = 4.5 b) x) =9 

13-21 at t = 3, x* ~ 2.79 

13-23 (a) fi(3 +A) = 334201 

(b) (3 +A) = 33 + 202 + 67 

13-25 (a) f:((0, 2) + A(1, —1)) = 24 — 34a 

(b) f2((O, 2) +A, -1)) = 24 — 342 +4 1102 

13-27 (a) stationary (c) Ax = (—16, 3) 

13-28 (a) definitely local minimum (c) definitely 
neither (e) possibly local maximum and possibly 
local minimum (g) possibly local minimum 

13-29 (a) concave (c) neither (e) both (g) convex 
(i) convex 

13-30 (a) global minimum 

13-31 (b) Ax = (23, 97) (c) max 100 — 5[(1 + 23d) — 
2}* — 3[G + 97a) — 5]* + 1 + 234) + 97A), st. 
A> 0) A = 0.027, x = (1.621, 5.619) 

(e) x® = (2.627, 5.435), x® = (2.675, 5.592) 


13-33 (a) fx((3, 7) + Ax) 2 68 + (—13, —93)Ax 4 
~60 1 

al 

Let Jo 


(b) Ax = (—0.2275, —0.6474) 
(c) x = (2.773, 6.353), x® = (2.681, 5.942) 
13-35 (a) Ax = (3, 98) 
(b) x = (2.078, 5.548), 
—1.003 —0.0268 
—0.0268 —0.0267 


13-37 (a) the function is not differentiable 
everywhere (b) x = (5, 2.5), x = (0, 1.25), 
x® = (—1.25, —0.9375), x® = (—1.5625, —0.2344) 


D= ).as = (9.595,,0.151), 
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13-39 (a) y? = (1,2, 1), y° = G, 3, 3), 
y® = (2,2,4),y® = 4, 2,4) 


Chapter 14 


14-1 (a) max wh, s.t. w+ 2h < 30, €+2h < 40, 
£,w,h = 0 (b) £* = 28.685, w* = 18.685, h* = 5.657 
14-2 (b) 2* = 27.386, w* = 27.386, h* = 66.667, 

d* = 52.974 

14-3 (a) min >, (Ajxj/2 + syd;/x;), st. 

a vjx; < 4000, x1, ...,x%5 > 0, where s;, v;, hj, and 
d; are the values in the table. 

(b) x* = (41.423, 55.937, 47.602, 52.600, 27.441) 
14-4 (b) x* = (10.338, 9.456, 6.304, 8.189, 3.740) 
14-5 (a) min 

52[42/(60N f) + 0.1(42/(5°9°7 N->-667 f-3))] + 
87[42/ (5° N->-57F-3)], s.t. 200 < N < 600, 

0.001 < f < 0.005 (b) N* = 200, f* = 0.001 

14-6 (b) x* = (83.052, 26.263, 

37.142, 31.391, 74, 284) 

14-7 (a) min YY}, wi (Gi — x? + ( — YP, 
Big gee Git = Ly cs DD Wee CU, 

fea ee Wwie0.t = 1,...,5,7 = 1,2, where d;,.e; 
and n; are the values in the table. (b) x} = 20, 

yj = 17, x5 = 4, yz = 30, wi, = 10, wi, = 50, 

W5. = 90, w3, = 35, wi, = 85, we, = 80 

14-8 (b) x* = 1.897, y* = 7.196, 

w* = (10, 0, 20, 40, 30) 

14-9 (a) min, pias VijXixj, St Pe = 1.5, 
ee mx; > (10)(1.5), x1, %2,.x3 => 0, where the m, 
are the mean return rates and v;; the covariances in 
the given table. (b) x} = 0.282, x3 = 1.218, x3 = 0.0 
14-10 (b) x* = (0.2, 0.6, 0.2) 


5 

14-11 (a) min Sg Cis. ei S15 

j=l 
12 < jy a1yxj S16, 31 < Fa xj < 36, 
Ind21) <In(, a3,x;) < In(164), x1,...,x5 > 0, 
where the c; are the costs, and a;; the ingredient 
index values in the table. 
(b) x* = (0.778, 0, 0.222, 0, 0) 
14-12 (b) x* = (0, 0, 1217.742, 0, 32.288) 
14-15 max ya Pisiri ep) s.t. pee eu aie ej 
(OS <o56, sec Ly pein s 5a) 


14-21 min fa(711, Tia, Thi, T >, Fy, Fy) +f3(S), s.t. 
Tho = fir, Fo, S, To); Ti — Tha) = A; 

Tia - Tah + kf, Tie, T21, Tha. Fi, Fx) = 
(To = Th2)Fo3 F;; = Ti < Taj, = 1, 2; 

NP (esse) et eee ee IU) 

14-23 (a) yes (c) no (e) yes 

14-24 (a) yes (c) yes (e) no 


14-25 (a) not QP (c) QP: ec = (0, 0, 9), 


O <0 
Oe leaks 
Ox20=20 


14-26 (a) no (c) yes 

14-27 (a) min 13x1x2(x3)~! + 9x1)! (43), st. 

3x (x3) + 8x2 (x3)! < 1, 5(%3)* < 1, x1, x2, x3>0; 
Ko = {1, 2}, Ky = {3, 4}, Kz = {5}, dy = 13, db = 9, 
dz = 3, ds = 8, ds = 5, 4,1 = 1, a2 =1,a,3 = —1, 


1 1 
21 = 3,42 = 0, 423 = 5, 431 = 1, 432 = 0, 
43,3 = —1, a4, = 0, a42 = 1, 443 = —1, a5, = 0, 
45,2 = 0, 453 = —4 


14-28 (a) 8(x, — 2)? + 2(a — 1)? 

+v(126 — 32x, — 12x) (b) 16(x, — 2) — 32v = 0, 
4(xg — 1) — 12v = 0, 32x, + 12x. = 126 (c) xf = 3, 
xh (e) = 2s = S67 

14-30 (a) all given primal constraints, plus 

2v; + 6v2 + v3 = 28(x, — 9), 

18v, + 82 +4 = 6(x2 Fe 5), 

—v, + 32 = 2(%3 — 11), v2 < 0, v3, v4 > 0, 

v2(20 — 6x1 — 8x2 — 3x3), v3(—x1) = 0, 

v4(—X2) = 0 (ec) all given primal constraints, plus 
(10x; — 1)]v1 + v3 = —2(%1 — 3), 

[60(x2 — 2)]v1 + 60v2 + v4 = —4(x2)3, 

39v2 + v5 = 19, V1, V3, V4, V5 > O, 

v1 [35 — S(x1 — 1)? — 30(x2 — 2)7] = 0, v3(—x1) = 0, 
v4(—Xx2) = 0, v5(—x3) = 0 

14-31 (a) yes (c) no 

14-32 (a) all given primal constraints, plus 

3y, + v2 = 30x), 2v; + v3 = 8x2, v2, v3 = 0, 
V¥2(—x1) = 0, v3(—x2) = 0 (e) vi = 10, v5 = v3 = 0 
14-34 (a) min 14(x, — 9)? + 30 —5)? + (4% 

11)? + (2x, + 18x. — x3 — 19] + max{0, 6x; + 
8x2 + 3x3 — 20} + max{0, —x,} + max{0, —x>}) 

(c) min 100 — (x, — 3) — (x2)* + 19x; 
+(max{0, 35 — 5(x, — 1)? — 300 
39x3 — 159| + max{0, —x,} + max{0, 
max{0, —x3}) 


2)7} + |60x2 + 
X2} 4 


14-35 (a) min 14(x, — 9)? +30 — 5)? +3 — 

11)? + po(|2x, + 18x. — x3 — 19]? + max?{0, 6x, + 
8x2 + 3x3 — 20} + max?{0, —x,} + max*{0, —x>}) 
(c) min 100 — (x, — 3)? — (x2)4 + 19x3 

+je(max?{0, 35 — 5(x, — 1)? — 30(x2 — 2)7} + [60x2 4 
39x53 — 159|? + max?{0, —x1} + max?{0, —x.} + 
max?{0, —x3}) 

14-36 (a) min 2(x; — 3)? — x)xX2 + (x2. — 5)? 
+p(max{0, (x1)? + (x2)? — 4} + max{0, x, —2} + 
max{0, —x,} + max{0, —x}) (b) original model was 
convex program and convexity is preserved by the 
chosen penalty functions (c) no (d) yes (e) penalty 
multipliers should start relatively low and increase 


slowly 
(f) x = (3,5); with wp = 5, 
x = (3,229, 4.646); with uw = 1, 
x® = (2.457, 3.743); with jz = 2, 

= (1.841, 2.730); with w = 4, 
x = (1.147, 1.762); with x = 8, 
x® = (1.088, 1.678) 
14-37 (a) min 2(x, — 3)? — xx + (%» — 5)? 
+yu(max?{0, (x1)? + (x2)? — 4} + max?{0, x, — 2} + 
max*{0, —x,} + max{0, —x2}) (b) original model 
was convex program and convexity preserved by 
the chosen penalty functions (¢) yes (d) no (e) 
penalty multipliers should start relatively low and 
increase slowly 
(f) x = (3,5); with uw = 0.5, 
x = (1,449, 2.190); with w = 1, 
x = (1,308, 1.991); with uw = 2, 
x® = (1.214, 1.858); with = 4, 
x = (1.157, 1.777); with p = 8, 
x®) = (1.124, 1.730) 
14-40 (a) not applicable due to equality constraint 
(ce) max x; + 6/x, + 5(x2)* + w(in(35 — 4x; — 6x2) + 
In(x,; — 5) + In(x2)) (e) not applicable due to 
equality constraint (f) not applicable due to 
equality constraint 


14-41 (a) not applicable due to equality constraint 
(c) max x, + 6/x; + 5(x2)? — [1/5 — 4x1 — 6x2) + 
1/(q, — 5) + 1/(2)] (e) not applicable due to 
equality constraint 

14-42 (a) min 2(x, — 3)? — xx) + GQ — 5)? — 
po[ln (4 — (x1)? — (@2)?] + In — x1) + In@1) + In@) 
(b) original model was convex program and 
convexity preserved by the chosen barrier functions 
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for x > 0 (c) yes (d) no (e) barrier multipliers 
should start relatively high and decrease to 0 

(f) x© = 6,5); with w = 2,x = (0.991, 1.615); 
with p= $,x° = (1.059, 1.663); 

with p = ; »x® = (1.081, 1.674); 

with w= ti ,x = (1.086, 1.677) 

14-43 (a) 1 min 2 (x —23)? = xpx9--+ (eo 5)? 
u[1/(4 — Gay — @2)?] +.1/@2 — x1) +1 
(b) original model was convex program and 
convexity preserved by the chosen barrier functions 
for x > 0 (c) yes @) no (e) barrier multipliers 
should start relatively high and decrease to 0 

(f) x® = (3, 5); with w = 2,x™ = (0.978, 1.553); 


1/x2 


with w = 4,x® = (1.031, 1.614); 
with pp = 1.x = (1.061, 1.645); 
with x = 5.x“ = (1.075, 1.661) 


14-46 (a) min (x; — 8)? + 2(% — 4)?, st. 
2x; + 8x2 +3 = 16,4) 2X4 = 7, X41, X20, %3,%4 = 0 
(b) columns linearly independent (c) N,B,S,B 
(d) r = (—13. 0, 1.5. 0) (e) Ax = 
(13, —3.0625, —1.5, —13) (f) Amax = 0.32653, 

x') = (4.2449, 0, 7.5102, 2.7551) (g) basic x2 
dropped to 0; new basis {x3, x4} or {x1, x4} 


14-47 (a) x = (0,1, 8,7), 

ae (4.2449, 0, 7.5102, 2.7551); then either 
= (4.6393, 0.8402, 0, 2.3607), 

x* = x® = (6.2222, 0.4444, 0, 0.7778), or 

x® = (7,0, 2, 0), x® = (7, 0.25, 0, 0), 

x* =x —= (6.2222, 0.4444, 0, 0.7778) 


6 -3 0) 
14-50 (a) Q=] —-3 y) 0 |, 
0 0 4 
5 
athe 2) 
—16 
b= (b) 12x, t 6x2 t 1y t 3v2 = ) 
6x, — 4x2. + 3v, —1v2 = 15, —8x3 —2v; + lv. = —16, 
1x, + 3x2 — 2x3 = 2, 3x, — 1x. + 1x3 = 3 
(C) Xj = 35 = 34 = 1,9) =5, 5. — 2 


14-52 (a) cy = 96, ¢ = (16, 19.0=( : : ) 
a® = (2,8), a® = (1,0), a® = 1, 0), a® = ©, 1), 
b, = 16, b> = 7, b3 = 0, by = 0, G = (3, 4}, 

= {1,2}, E = % (b) —2 Ax, + 113 = —16, 
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—4 Ax, = —12,1 Ax, = 0; solution Ax; = 0, 

Ax, = 3, v3 = —16 (c) A = §, x = (0,2) 

(d) —2 Ax; + 2v, +113 = —16, —4 Ax, + 8v,; = —8, 
2 Ax; +8 Axp =0, Ax; = 0; solution Ax, = 0, 
Axy = 0,1 = 14; Ax = 0 implies no 
further progress (e) drop i = 3, x; > 0 

(f) x® = (0,2), x® = x* = (6.222, 0.444) 


14-54 (a) min 1 + x11/3 + 4x%12/3 + 4x29, s.t. 
2x11 ste 2X19 =F 1x21 + 1x22 Ss 

4 + 28x14 + 112x12 18x21 54x 9 = 25, 
O< x41 < 10<x12 <2,0< x4 <p) 

O< x22 <2 


1,43 = 


14-55 (b) unconstrained minimum is feasible 
(c) min 18 — 8x, + 4x0, s.t. 0 SX, +X. < 6, 


O< x, <2,0 <x <4) xj =2, x5 = 0; sequence 
correct (g) xj = 0, x} = 4; sequence incorrect 


14-56 (a) min 3 exp(—0.5z)) + exp(z1 + Z2) 

+10 exp(—3z3), s.t. 0.5 exp(z; + z2 — 2z3) <1, 
0.167 exp(zi)+0.25 exp(.4z1 +22) +0.0833 exp(z3) < 
1,z URS (b) z* = (1.360, —31.1, 1.433), 

x* = (3.898, 3 x 10-4, 4.191) (c) max 

6; In(3/61) + 2 In(1/d2) + 43 In(10/83) + 

34 In(0.5/54) + 55 In(0.167/85) + 5¢ In(0.25/8¢) + 

57 In(0.0833/67) — vy In(—v;) — v2 In(—v), s.t. 
—0.55; + 6. + 64 + 55 + 0.456 = 0, 52 + 54 + 65 = 0, 
—353 — 264 + 67 = 0, 6; + 62 +63 = 1, 64 = Vil 

65 + 56 + 57 = —V2, 61,...,67 = 0,14, vo <0 (d)4 
(e) 5* = (0, 0, 1, 0, 0, 0, 3), v* = (0, —3), x* as in 
part (b) 


Index 


* notation, 5 

# notation, 652 

V notation, 43 

[ notation, 640 

| notation, 640 

v notation, 421, 425, 656 

] notation, 640 

| notation, 640 

0-1 variables (see discrete 

variables) 

1-dimensional search, 726-737 
bracketing, 731-734 
golden section search, 727-731 
quadratic fit, 734-737 

3-point patterns, 731-734 
Algorithm 13B, 733 


A 


A or q;,; notation, 184 
AA Crew Scheduling, 572 
AAx = 0 (see net change zero 
condition) 
absolute value objectives (see 
minimum deviation objectives) 
active constraints, 105 
conditions for feasible 
directions, 107-109 
active partial solutions, 653 
active sct methods, 852-858 
Algorithm 14D, 856 
KKT conditions, 852, 854 
step sizes, 853-854 
updating the active set, 855-856 
activities 
implicit pricing, 319-320 
interpretation of decision 
variables, 302 
project management, 450 


acyclic digraph shortest paths, 
449-450 
Algorithm 9D, 449 
acyclic digraphs, 446-448 
project networks, 458 
adjacent extreme points, 187 
AEL, 168 
affine scaling, 262-267 
diagonal matrix formalization, 
264-265 
standard form, 266-267 
affine scaling search, 269-275 
Algorithm 6A, 274 
directions, 269-271 
dual-based termination, 289 
finite termination, 273 
Agrico Chemical, 517 
all or nothing constraints, 555-557 
allocation decision variables, 132 
allocation models, 131-135 
allocation decision variables, 
132 
alternative optimal solutions, 
34-35 
American Airlines, 572 
American Edwards Laboratories, 
168 
American Olean, 552 
anti-cycling rules, 225 
approximate optimal solution 
(see heuristic optimal solutions) 
arcs, 411, 475 
capacities, 477 
costs, 477 
flows, 475 
lengths, 413 
artificial network flow model, 
498-499 
artificial variables, 119-120 


assignment constraints, 506, 575 
assignment problems 
generalized, 580-582, 584 
linear, 506-510, 575-576, 584 
matching, 582-584 
quadratic, 576-580, 584 
(see also each model class) 
asymmetric traveling salesman 
problems (see traveling 
salesman problems) 
ATT, 595 
Ault Food Limited, 621 


B 


b or b; notation for RHSs, 184 
B notation for basis matrix, 227 
B~! notation for basis matrix 
inverse, 227 
backwards dynamic 
programming, 466-467 
balance constraints, 145-147, 477 
network flow, 477 
operations planning, 145-148 
time-phased models, 154-155 
balance of flow, 477 
Bank Three, 373 
barrier functions, 833-834 
barrier methods 
LPs, 275-287, 291-293 
NLPs, 833-836 
sequential unconstrained 
barrier technique, 836 
(see also Newton step barrier 
search) 
(see also primal-dual barrier 
search) 
barrier multipliers, 275, 833 
management, 284, 835 


905 


906 


Index 


barrier objective function, 275, 
833 
basic feasible solutions, 194 
equivalence with extreme 
points, 194-197 
minimum cost flows, 524-525 
via two-phase simplex, 215 
(see also basic solutions) 
basic solutions, 189-197 
existence, 190-192 
in lower- and upper-bounded 
form, 239-240 
minimum cost flows, 524-525 
(see also basic feasible 
solutions) 
basic variables, 189, 838 
basis, 194 
finite number, 224-225 
linear independence, 194 
minimum cost flows, 519-524 
reduced gradient, 838-840, 
844-845 
update in lower- and 
upper-bounded simplex, 
240-243 
update in simplex, 203-204 
basis inverse, 227 
B* notation, 227 
product representation, 
230-233 
basis matrix, 227 
B notation, 227 
Bay Ferry, 537 
Beer Belge, 787 
Bellman-Ford shortest paths, 
426-432 
Algorithm 9A, 428 
negative dicycles, 432-433 
benefits, 300 
best first search, 671 
versus depth first, 671-675 
versus depth forward best 
back, 671-675 
beta distribution, 722 
Bethlehem Steel, 52 
BFGS formula, 768 
approximation to Hessian 
inverse, 774 
BFGS search, 768-774 
Algorithm 13F, 770 


big-M constants, 124, 642-644 
big-M method, 124 
artificial model, 124 
objective function, 124 
outcomes, 125-127 
binary variables (see discrete 
variables) 
binding constraints (see active 
constraints) 
Bison Boosters, 631 
blending models, 135-139, 
789-792 
composition constraints, 
136-138 
ingredient decision variables, 
135-136 
LP, 135-139 
NLP, 789-792 
Blue Bell, 170 
Blue Hills Homes Corporation, 
71 
boldface notation, 79 
boundary points, 177-178, 252 
optimal for LPs, 179 
bounds 
from duality, 289-290, 329-333 
from relaxations, 635-637 
bracketing (see 3-point patterns) 
branch and bound, 651-676 
candidate problems, 656-658 
enumeration sequences, 
671-676 
error bounds, 670-671 
global optimality, 656 
incumbent solutions, 655 
LP-based, 660-663 
partial solutions, 651-652 
stopping early, 670 
terminating by bound, 658 
terminating by parent bound, 
669 
terminating by solving, 659 
terminating infeasible, 658 
tree search, 652-655 
branch and bound trees, 652 
branch and cut, 676-680 
Algorithm 12B, 677 
valid inequalities, 676 
(see also branch and bound) 
branching rules, 654, 661 
Brisbane, 70 
budget constraints, 563-564 


C 


¢ or c; notation, 184 
¢ or c; notation, 200 
CAM Assignment, 507, 575 
Canadian Department of 
Transportation, 580 
Canadian Forest Products 
Limited, 141 
candidate problems, 657-658 
capital budgeting problems, 
562-566 
budget constraints, 563-564 
dependent projects, 565 
mutually exclusive choices, 
564-565 
cash flow models, 153-157, 
541-542 
CDOT, 580, 647 
ceiling [ ] notation, 640 
CFPL, 141 
chains, 483 
versus paths, 484 
child of a tree node, 668 
Clever Clyde, 211 
closed form solutions, 11 
Cofferdam Design, 805 
column generation methods, 
572-574 
combinatorial optimization (see 
discrete optimization) 
complementary slackness 
dual LP, 323-324 
primal LP, 322-323 
primal NLP, 818 
complete enumeration (see total 
enumeration) 
completion time, 605 
completions of partial solutions, 
652, 657 
components of vectors, 78-81 
subscript notation, 79 
composition constraints, 136-138 
compositions of convex/concave 
functions, 754 
concave functions, 750-752 
sufficient for global maxima, 
752-753 
tests, 753-756 
versus unimodal, 756-757 
conflict constraints, 603, 611-612 
connected graphs, 522 
conservation of flow (see balance 
of flow) 


constrained nonlinear 
programming 
active set methods, 852-858 
barrier methods, 833-836 
blending models, 789-792 
convex programs, 797-800 
differentiable, 817-818 
engineering design models, 
792-795 
geometric programs, 805-810, 
866-874 
Karush-Kuhn-Tucker 
conditions, 817-827 
Lagrange multiplier methods, 
810-817 
linearly constrained, 789-792 
location allocation problems, 
787-789 
penalty methods, 827-833 
quadratic programs, 803-805, 
849-858 
reduced gradient methods, 
837-848 
separable programs, 801-803, 
858-866 
constraint coefficients, 184 
A or a;,; notation, 184 
interpretation, 302-303 
sensitivity analysis, 307-311 
constraint matrix, 184 
A notation, 184 
constraint qualifications, 826 
constraint relaxations, 632 
(see also relaxations) 
constraints, 4, 24-25 
adding and dropping, 308-309, 
350-352 
interpretation, 300-301 
relaxing versus tightening, 305 
sensitivity analysis, 304-311, 
344-346 
sensitivity outputs, 339 
construction costs (see fixed 
charges) 
constructive search, 698-705 
algorithm 12G, 699 
greedy rules, 699-701 
continuous improving search 
Algorithm 3A, 96 
(see also improving search) 
continuous optimization model, 
56 


continuous relaxations, 633 
continuous variables, 53 
versus discrete, 54, 144 
contours of objective functions, 
32 
contraction step in Nelder-Mead, 
7719 
convex feasible sets, 113-116 
linearly constrained, 116-117 
nonlinearly constrained, 800 
tractability for improving 
search, 117-118 
convex functions, 750-752 
sufficient for global minima, 
752-753 
tests, 753-756 
versus unimodal, 756-757 
convex programming, 797-800 
global optima, 800 
separable, 864-865 
sufficiency of KKT conditions, 
825 
corner points, 177 
costs, 300 
(see also objective functions) 
covariances of returns, 804 
covering constraints, 567 
in shift scheduling, 151-153 
CPM (see critical path methods) 
crew pairing, 572 
criterion functions (see objective 
functions) 
critical path methods, 451-457 
Algorithm 9E, 455 
versus longest paths, 453 
critical paths, 453-454 
crossover in genetic algorithms, 
695 
curve fitting, 718-721 
linear versus nonlinear, 
719-720 
Custom Computer Company, 718 
Custom Metalworking, 609 
cutting planes, 644 
cycle canceling search, 529-536 
Algorithm 10C, 534 
feasible directions, 530-531 
Floyd-Warshall algorithm, 
532-533 
improving directions, 531-532 
residual digraphs, 529-530 
cycle direction search, 494-500 
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Algorithm 10A, 495 
cycle canceling, 529-536 
network simplex, 519-529 
starting feasible solutions, 
497-499 
step sizes, 491 
cycle directions, 484-486 
computational convenience, 
499-500 
feasible, 487-489 
Floyd-Warshall algorithm, 
532-533 
improving, 490-491 
preserve flow balance, 486-487 
residual digraphs, 530-532 
simplex, 525-526 
step sizes, 491-492 
sufficient for optimality, 
492-493 
cycles, 483 
versus dicycles, 484 
cycling with degeneracy, 225 


D 


DClub, 81 
decision problems, 3 
decision variables, 5, 24 
activity interpretation, 302 
adding and dropping, 314-315, 
352-354 
implicit pricing, 319-320 
objective sensitivity, 311-314, 
349-350 
decisions, 4 
in dynamic programming, 460 
decrease arc, 529 
decreasing returns to scale, 51 
deficiency variables, 391 
deflection matrices, 766-767 
degeneracy, 215, 220 
cycling, 225 
difficulties for simplex, 220-224 
degrees of difficulty, 872 
demand nodes (see sinks) 
demands, 301, 478 
dependent projects, 565 
depth first search 
acyclic digraphs, 447-448 
branch and bound, 654 
versus best first, 671-675 
versus depth forward best 
back, 671-675 
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depth forward best back search, 
671-675 
derivatives, 100, 725 
descriptive models, 12 
versus prescriptive, 12-14 
determinants of matrices, 193 
checking convexity/concavity, 
754 
checking singularity, 193 
deterministic models, 16 
versus stochastic, 18-19 
diagonal matrices, 264 
dicycles, 420 
versus cycles, 484 
(see also negative dicycles) 
differentiable nonlinear 
programming, 817 
(see also constrained nonlinear 
programming) 
digraphs, 415, 475 
acyclic, 446-448 
Dijkstra shortest paths, 440-446 
Algorithm 9C, 440 
dimension (see vectors) 
directed graphs (see digraphs) 
directing a graph, 415-416 
direction-step paradigm, 88-90 
directions (see move directions) 
discrete dynamic programming 
(see dynamic programming) 
discrete improving search, 
680-685 
Algorithm 12C, 681 
move sets, 680-684 
nonimproving moves, 687 
(see also improving search) 
discrete optimization, 56 
branch and bound search, 
651-676 
branch and cut search, 676-680 
capital budgeting models, 
562-566 
facility location, 594-598 
fixed charge problems, 558-560 
generalized assignment 
problems, 580-582, 584 
job shop scheduling, 609-613 
knapsack problems, 561-562 
linear assignment problems, 
575-576, 584 
lumpy linear programs, 
556-560 


matching, 582-584 
modeling as continuous, 144 
network design, 598-601 
quadratic assignment 
problems, 576-580, 584 
routing, 584-594 
set covering, packing, 
partitioning, 566-574 
single processor scheduling, 
602-609 
total enumeration, 627-630 
traveling salesman problems, 
584-592 
(see also each model class) 
discrete variables, 53 
versus continuous, 54, 144 
disjunctive constraints, 604 
big-M constants, 642-644 
disjunctive variables, 603 
dot product of vectors, 80-81 
dual complementary slackness, 
323-324, 335-336 
dual constraints, 320-321 
dual variables, 316 
price interpretation, 318-319 
V or v; notation, 233-234 
variable types, 317-318 
duality, 316 
bounds on primals, 289-290, 
329-331 
by-product optimum, 332-334 
complementary slackness, 
322-324, 335-336 
equality with primal, 321-322, 
331-335 
formulating, 324-328 
geometric programming, 
870-874 
infeasible primals, 336-338 
of the dual, 328 
standard form, 288 
strong, 321-322, 331-335 
unbounded primals, 336-338 
weak, 289-290, 329-331 
(see also dual constraints) 
(see also dual variables) 
dualized constraints, 647 
due dates, 602, 605 
DuPage County, 60-62 
durations, 450 
dynamic models (see 
time-phased models) 


dynamic programming, 417 
approach to shortest paths, 
417-426 
backward solution, 466-467 
digraph, 460-461 
functional equations, 421-426, 
461-462 
functional notation, 418 
knapsack problems, 464-466 
modeling, 462-466 
multiple solutions, 468 
principle of optimality, 421-422 
stages, 463 
states, 459 


E 
E-Mart, 50 
early start schedules, 450 
Algorithm 9E, 455 
longest paths, 453 
early start times, 450 
economic order quantity, 9 
edges 
of a graph, 411 
of the LP-feasible space, 188 
efficient frontier, 381-382 
constructing, 382-384 
efficient points, 379-381 
with goal programs, 397-400 
with preemptive optimization, 
386-387 
with weighted sums, 389 
efficient solutions (see efficient 
points) 
elite solutions, 696 
EMS, 566, 636 
engineering design models, 
792-794, 805-807 
Environmental Protection 
Agency, 170 
EOQ (see economic order 
quantity) 
EPA, 170 
equal returns to scale, 51 
error bounds, 670-671 
Euclidean distance, 162 
versus rectilinear, 162 
exact optimal solutions, 15 
versus heuristic, 16 
(see also global optima) 
exact penalty functions, 830-831 
expansion step in Nelder-Mead, 
779 


exponential growth, 629 

{otal enumeration, 629-630 
extreme points, 177-178 

adjacent, 187 

equivalence with basic feasible 
solutions, 194-197 

optimal in LPs, 179 


F 


facility layout models, 577 
facility location problems, 
594-598 
big-M constants, 642-644 
switching constraints, 596 
fathoming (see termination) 
FDA, 405 
feasible completions, 652 
feasible directions, 92-94 
active constraint conditions, 
107-109 
cycle, 487-489, 530-531 
KKT connection, 822-825 
linearly constrained NLPs, 838 
feasible models, 29, 35-37 
concluding with big-M@ 
method, 125 
concluding with two-phase 
method, 122 
feasible region (see feasible set) 
feasible set, 27 
convex, 113-117, 800 
graphing, 27-29 
feasible solutions, 5 
feasible space (see feasible set) 
Filter Tuning, 837 
finish node, 451 
first derivatives, 100, 738 
first partial derivatives (see 
gradients) 
first-order necessary optimality 
conditions, 744 
first-order Taylor approximation, 
741-743 
fixed charge network flows (see 
network design problems) 
fixed charge problems, 558-560 
big-M constants, 642-644 
facility location, 594-598 
fixed charge variables, 558 
network design, 598-601 
switching constraints, 558 
fixed charges, 558 


versus variable, 558 
fixed time horizons, 156 
versus infinite, 156-157 
floor | | notation, 690 
flow balance constraints, 477 
(see also balance constraints) 
flow time, 605 
flow-distance, 577 
flows with gains and losses (see 
gain/loss flows) 
Floyd-Warshall shortest paths, 
433-438 
Algorithm 9B, 434 
in cycle canceling, 532-533 
negative dicycles, 439-440 
Food and Drug Administration, 
405 
for all, for every VY notation, 43 
forward arc, 484-486 
fractional variable branching 
rule, 661 
Frannie’s Firewood, 252 
free variables, 652 
# notation, 652 
frequency histogram, 17 
functional equations, 421-426, 
461-462 
all to all shortest paths, 425-426 
knapsack problems, 464-466 
one to all shortest paths, 
421-425 
Wagner-Whitin lot sizing, 
461-462 
functional notation, 418 


G 


gain/loss flows, 541-543 
tractability, 543 
GAP (see generalized 
assignment problems) 
Gaussian elimination, 207 
generalized assignment 
problems, 580-582 
Lagrangian relaxations, 
647-649 
tractability, 584 
generalized reduced gradient 
methods, 848 
generations in genetic 
algorithms, 695 
generic interpretation of LPs, 
299-304 
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genetic algorithms, 694-697 
A\gorithm 128, 696 
crossover, 695 
elite solutions, 695 
generations, 695 
immigrant solutions, 695 
solution encoding, 696-697 

geometric programming, 
807-810, 866-874 
(see also posynomial 
geometric programming) 

global maxima, 85 
concave sufficient, 752-753 
(see also global optima) 

global minima, 85 
convex sufficient, 752-753 
(see also global optima) 

global optima, 85 
from branch and bound, 656 
from convex programs, 800 

from improving search, 118 
versus local optima, 85-87 
goal levels, 390 
goal programs 
deficiency variables, 391 
efficient points, 397-400 
minimum deficiency objective, 
392-394 
modeling soft constraints, 
391-392 
preemptive, 394-396 
versus other multiobjective, 
397 

golden ratio, 728 

golden section search, 727-732 
Algorithm 13A, 729 

GP (see geometric 

programming) 

gradient norm, 758 

gradient search, 757-761 
Algorithm 13D, 757 
zigzagging and convergence, 
761 

gradients, 99-100, 738-739 
as move directions, 104-105, 
252-253, 278, 757-761 
conditions for improving 
directions, 102-104 
graphic interpretation, 101 

graphic solution, 27-39 
constraints, 27-29 
feasible set, 28-29 
infeasible models, 35-37 
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objective functions, 30-33 

optimal solutions, 33-35 

unbounded models, 37-39 
graphs, 411 

connected, 522 

(see also digraphs) 

(see also undirected graphs) 
greedy heuristics, 699 


H 


Hanshin expressway, 172 

hard constraints, 391 

Hessian matrices, 739-741 

heuristic optimal solutions, 

16, 680 
versus exact, 16 

heuristic optimization 
constructive search, 698-705 
genetic algorithms, 694-698 
improving search, 88, 680-685 
multistart, 685-686 
simulated annealing, 690-694 
tabu search, 687-690 
truncated branch and bound, 
670-671 

Highway Patrol, 158 

hillclimbing (see improving 

search) 

holding arcs, 517 

Homart, 171 

how far (see step sizes) 

Hungarian algorithm, 509 


I 


I notation for identity matrices, 

228 

identity matrix, 228 

IFS, 153 

ILP (see integer linear 

programming) 

immigrant solutions, 696 

implicit prices (see dual 

variables) 

improving directions, 90-92, 737 
cycle, 490-491, 532-533 
from gradients, 104-105 
gradient conditions, 102-104 
KKT connection, 822-825 
NLP deflection matrix 
conditions, 768 

improving search, 83-85, 737-738 


active constraint conditions for 
feasibility, 107-109 
cases with local optima global, 
118 
continuous Algorithm 3A, 96 
detecting unboundedness, 
98-99 
directions feasible, 92-94 
directions improving, 90-92 
direction-step paradigm, 88-89 
discrete, 680-685 
discrete Algorithm 12C, 681 
gradient conditions for 
improvement, 102-103 
initial solutions, 78, 118-127 
neighborhoods, 84-85 
step sizes, 94-95 
stopping with local optima, 
96-97 
tractability, 88, 109-118, 
176-177 
with convex feasible sets, 
117-118 
with unimodal objective 
functions, 109-112 
improving scarch heuristics, 
680-698 
inactive constraints (see active 
constraints) 
increase arc, 529 
incumbent solutions, 656 
D notation, 656 
as heuristic optima, 670, 686, 
688, 691 
indexing, 40-45 
decision variables, 40-41 
families of constraints, 43-45 
making models large, 45-46 
summations, 42 
symbolic parameters, 41-42 
time, 154-155 
Indy Car, 561 
infeasible models, 35-37 
concluding from big-M 
method, 126 
concluding from two-phase 
method, 123-124 
detecting with simplex, 
214-217 
primal to dual relationships, 
336-338 
proving with relaxations, 


634-635 
infinite time horizons, 156 
versus fixed, 156-157 
ingredient decision variables, 
135-136 
initial costs (see fixed charges) 
INLP (see integer nonlinear 
programming) 
Institutional Food Services, 153 
integer linear programming, 57, 
63 
multiobjective, 376-378 
(see also discrete optimization) 
integer nonlinear programming, 
57, 63 
(see also discrete optimization) 
integer programming, 56 
(see also discrete optimization) 
integer variables (sce discrete 
variables) 
integrality property 
lacking in gain/loss flows, 543 
lacking in multicommodity 
flows, 540 
minimum cost flows, 500-501 
interior point methods, 251-269 
affine scaling search, 269-275 
computational burden, 268-269 
log barrier search, 275-287 
Newton step barrier search, 
278-287 
primal-dual barrier search, 
291-293 
projective scaling search, 251 
strategy, 253-255 
interior points, 177-178, 252 
convenience for feasible 
directions, 252-254 
in LP standard form, 255-256 
inventory models, 2-8, 156-157, 
517-519, 795-797 
inverse affine scaling, 262-266 
inverse of a matrix, 228-229 
IP (sce integer programming) 


J 


job shop scheduling, 609-613 
conflict constraints, 611-612 
precedence constraints, 

610-611 
(see also scheduling) 
jobs, 507 


joint probability density 
function, 721 


K 


Karush-Kuhn-Tucker conditions, 
819-821 
active set methods, 852-858 
improving feasible direction 
connection, 821-825 
necessity, 826-827 
quadratic programs, 850-851, 
852-855 
sufficiency for convex 
programs, 825 
KI Routing, 592, 703 
KKT conditions (see 
Karush-Kuhn-Tucker 
conditions) 
KKT point, 819 
knapsack problems, 561-562 
as dynamic programs, 464-466 
Kraft, Inc., 592 


L 


Lagrange multiplier techniques, 
810-817 
stationary point conditions, 
812-813 
Lagrange multipliers, 647, 810 
interpretation, 815-816 
sign restrictions, 818-819 
(see also dual variables) 
Lagrangian functions, 811 
Lagrangian relaxations, 647-650 
multipliers, 647, 650 
tractability, 649 
Lancaster, 69 
large-scale optimization models, 
39-46 
late start schedules, 456-457 
late start times, 456 
lateness, 605 
lead times, 5 
learning curves, 782 
least squares curve fitting, 257, 
720 
left-hand sides 
interpretation, 302-304 
(see also constraint 
coefficients) 


lexicographic goal programming 
(see preemptive goal 
programming) 
lexicographic optimization (see 
preemptive optimization) 
LHS (see left-hand sides) 
likelihood functions, 721-722 
line search (see 1-dimensional 
search) 
line segments, 113 
algebraic characterization, 
115-116 
linear assignment problems, 
506-510, 575-576 
integrality, 508-510 
standard form, 509 
tractability, 584 
versus matching, 582 
linear combinations, 194 
(see also linearly independent 
vectors) 
(see also weighted sums) 
linear constraints, 48, 107 
conditions for feasible 
directions, 107-108 
convexity of feasible set, 
116-117 
linear equations, 193-194 
linear functions, 48 
convex and concave, 754 
equal returns to scale, 51 
linear objective functions, 111 
unimodal, 111-112 
linear program-based branch and 
bound (see LP-based branch 
and bound) 
linear programming, 49, 64, 131 
allocation models, 131-134 
blending models, 135-139 
constraints as resources, 
300-301 
degeneracy, 220-225 
dual bounds, 289-290 
formulation of duals, 324-327 
generic interpretation, 299-304 
inequalities as supplies and 
demands, 300-301 
linearizable nonlinear 
objectives, 157-164 
lower- and upper-bounded 
simplex search, 238-245 
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modeling integer quantities, 
144 
multiobjective, 60-62, 373-375 
objectives as costs and 
benefits, 300 
operations planning models, 
139-148 
primal versus dual, 315-316 
qualitative sensitivity, 304-315 
quantitative sensitivity, 
315-324 
relaxations of ILPs, 633-634 
revised simplex search, 
225-228 
shift scheduling and staff 
planning models, 148-153 
simplex search, 197-207 
standard forms, 180-186, 
238-239, 266-267 
step sizes, 201 
terminating optimal, 217 
testing infeasibility, 214-217 
testing unboundedness, 219 
time-phased models, 153-157 
tractability, 176-177, 426 
variables as activities, 302 
(see also duality) 
linear programming relaxations, 
633-634 
linear regression (see curve 
fitting) 
linear Taylor approximation (see 
first-order Taylor 
approximation) 
linearly dependent vectors, 194 
impossible in a basis, 194 
relation to singularity, 194 
linearly independent vectors, 194 
relation to nonsingularity, 194 
versus bases, 194 
links (see edges) 
Littleville, 409 
local improvement (see 
improving search) 
local maxima, 85 
(see also local optima) 
local minima, 85 
(see also local optima) 
local optima, 85 
caused by constraints, 112 
caused by objective functions, 
110-112 
first-order necessary 
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conditions, 744 
from improving search, 85, 
87-88, 737 
KKT conditions, 825-827 
second-order necessary 
conditions, 748 
second-order sufficient 
conditions, 748-749 
single variable search, 726-727 
versus global optima, 85-87 
(see also improving search) 
location models, 81, 787 
location-allocation models, 
787-789 
log barrier functions, 275-278 
log barrier methods (see barrier 
methods) 
logistics curve, 782 
longest path problems, 458 
tractability, 458 
(see also shortest path 
problems) 
lot sizing, 459-462, 795-797 
lower- and upper-bounded 
simplex search, 238-245 
Algorithm 5D, 244 
basic solutions, 239-240 
step sizes, 241-242 
LP (see linear programming) 
LP-based branch and bound, 
660-664 
Algorithm 12A, 660 
branching rules, 661 
(see also branch and bound) 
lumpy linear programs, 556-560 
all or nothing constraints, 
555-557 
fixed charge problems, 558-560 


M 


main constraints, 25 
major iterations in reduced 
gradient, 847 
Major League Baseball, 552 
makespan (see maximum 
completion time) 
Mall Layout, 577 
marginal prices (see dual 
variables) 
matching problems, 582-583 
tractability, 584 
versus linear assignment, 582 


material balance constraints (see 
balance constraints) 
mathematical programs, 4 
standard form, 26, 46-47 
matrices, 185 
boldface upper case notation, 
185 
maximal sets of vectors, 194 
maximin objectives, 159 
linearizing, 159-160 
maximum completion time, 605, 
607-608 
maximum flow problems, 513-516 
network flow formulations, 
515-516 
maximum flow time, 605, 607-608 
maximum lateness, 605, 607-609 
maximum likelihood estimation, 
721-723 
maximum tardiness, 605, 607-609 
maxisum objectives, 158 
mean completion time, 605, 608 
mean flow time, 605, 608 
mean lateness, 605, 608 
mean tardiness, 605, 608 
method of steepest ascent (see 
gradient search) 
method of steepest descent (see 
gradient search) 
minimax objectives, 159 
linearizing, 159-160 
scheduling, 605-608 
minimum cost flows, 477-483 
basic solutions, 524-525 
cycle canceling search, 529-536 
cycle direction search, 494-500 
cycle directions, 484-494 
formulation of maximum 
flows, 515-516 
formulation of shortest paths, 
513 
integer optima, 500-501 
linear dependence of cycles, 
519-522 
network simplex search, 
519-529 
node-arc incidence matrices, 
481-483 
residual digraphs, 529-530 
single commodity, 511 
spanning tree bases, 522-523 
standard form, 477, 481 


time-expanded models, 
517-519 
total supply = total demand, 
480-481 
tractability versus LP, 499-500 
tractability versus shortest 
paths, 512 
minimum deviation objectives, 
162 
linear modeling, 162 
minimum ratio rule (sce step 
sizes) 
minisum objectives, 158 
MINLP (see mixed-integer 
nonlinear programming) 
minor iterations in reduced 
gradient, 847 
MIP (see mixed-integer 
programming) 
mixed-integer nonlinear 
programming (see integer 
nonlinear programming) 
mixed-integer programming, 56 
(see also discrete optimization) 
Mobil Oil Corporation, 623 
Monsanto, 621 
Monte Carlo analysis (see 
stochastic simulation) 
Montreal, 622 
Mortimer Middleman, 2 
move directions, 88, 737 
affine scaling, 269-271 
BFGS, 767-770 
cycle, 484-486, 530-532 
gradient, 104-105, 252-253, 
278, 757-761 
Nelder-Mead, 777-778 
network simplex cycle, 525-526 
Newton step barrier, 279-280 
Newton’s method, 762 
primal-dual barrier, 291-292 
reduced gradient, 842-843 
simplex, 198-200, 240-241 
move sets, 680-685 
multi-dimensional knapsack (see 
capital budgeting problems) 
multicommodity flows, 536-540 
tractability, 540 
versus single commodity, 
536-538 
multiobjective optimization, 
59-63, 373-400 
efficient frontier, 381-382 


efficient points, 379-381 
engineering design models, 
375-376 
finance models, 373-375 
goal programming, 389-400 
preemptive, 384-386 
public sector models, 60-62, 
376-378 
versus single-objective, 63-64 
weighted sums, 387-389 
multistart search, 685-686 
(see also discrete improving 
search) 
mutually exclusive choices, 
564-565 
myopic rules (see greedy rules) 


N 


NASA, 562, 664, 699 
National Cancer Institute, 622 
NCB, 58, 682 
nearest child rule, 672 
negative definite matrices, 747 
local optimality conditions, 
748-749 
test for concave functions, 754 
negative dicycles, 420 
Bellman-Ford algorithm, 
432-433 
difficulty for shortest paths, 
420-421 
Floyd-Warshall algorithm, 
439-440 
improving cycle directions, 
532-533 
negative semidefinite matrices, 
747 
local optimality conditions, 
748-749 
test for concave functions, 754 
neighborhood search (see 
improving search) 
neighborhoods, 84-85, 738 
discrete, 680-684 
Nelder-Mead search, 774-781 
Algorithm 13G, 775 
directions, 777-778 
shrinking, 780-781 
step sizes, 778-779 
net change zero condition, 108, 
198, 210, 257, 486, 492, 838 


network design problems, 
598-601 
big-M constants, 642-644 
switching constraints, 601 
network flows 
gain/loss flows, 541-543 
linear assignment, 506-510 
maximum flow, 513-517 
minimum cost flows, 477-483 
multicommodity, 536-540 
network design, 598-601 
shortest path problems, 
512-513 
time-expanded models, 
517-519 
transportation problems, 
502-506 
(see also each model class) 
network simplex search, 519-529 
Algorithm 10B, 527 
simplex cycle directions, 
525-526 
network tableaux (see 
transportation tableaux) 
networks (see digraphs) 
Newton direction (see Newton 
step) 
Newton step, 278-279, 762 
Newton step barrier search, 
278-287 
Algorithm 6B, 285 
directions, 279-280 
dual-based termination, 290 
multiplier management, 
284-285 
step sizes, 281-284 
Newton’s method, 761-766 
Algorithm 13E, 764 
starting close, 766 
versus gradient search, 765 
Nifty Notes, 602 
NLP (see nonlinear 
programming) 
node-arc incidence matrices, 
481-483 
linear dependence of cycles, 
519-522 
spanning tree bases, 522-523 
total unimodularity, 501-502 
nodes, 411, 475 
nonbasic variables, 189, 838 
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superbasics in reduced 
gradient, 839 
nondegeneracy assumption, 841 
nondominated solutions (see 
efficient points) 
nonimproving moves, 687 
nonlinear constraints, 48 
nonlinear functions, 48 
nonlinear objective functions, 49, 
64 
linearizable cases, 157-164 
nonlinear programming, 49, 64, 
715 
multiobjective, 375-376 
(see also constrained nonlinear 
programming) 
(see also unconstrained 
nonlinear programming) 
nonlinear regression (see curve 
fitting) 
nonnegativity constraints, 25 
nonpositive variables, 183 
converting to nonnegative, 183 
nonsingular matrices, 193 
(see also singular matrices) 
nonsmooth functions, 724 
Nelder-Mead search, 774-781 
versus smooth, 724-726 
nonsymmetric matrices, 259 
norms of vectors, 80 
North American Van Lines, 173 
notation 
* for optimal solutions, 5 
# for free variables, 652 
V forall for every, 43 
[ and ] for round up, 640 
| and | for round down, 640 
v for optimal solution value, 
421, 425, 656 
A or a;,; for constraint 
coefficients, 184 
b or 5; for RHSs, 184 
B for basis matrix, 227 
B~! for basis matrix inverse, 
227 
boldface lower case for 
vectors, 79 
boldface upper case for 
matrices, 185 
¢ or c;,; for objective 
coefficients, 184 
¢ or c; for reduced costs, 200 
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functional in dynamic 
programming, 418 
I for identity matrices, 228 
italics for scalars, 79 
subscripts distinguish 
components, 79 
superscripts distinguish 
vectors, 79 
v for dual solutions, 233-234 
numerical search, 14 
dependence on starting point, 
15 


O 


objective function coefficients 
c of c; notation, 184 
multiple changes, 361-362 
ranges, 346-348 
sensitivity analysis, 311-314, 
349-350 
sensitivity outputs, 346 
objective functions, 25-26 
as move directions, 252-253 
contours, 32 
objective value space, 382 
objectives, 4 
Ohio National Bank, 148-149 
ONB, 148 
one-dimensional search (see 
1-dimensional search) 
OOI, 476 
operations, 507 
operations planning models, 
139-148 
balance constraints, 145-147 
Operations Research, 1 
operations research approach, 3 
dependence on time and 
resources, 19 
optima (see global optima) 
Optimal Ovens Incorporated, 
476 
optimal solutions, 5 
exact, 15 
from relaxations, 637-639 
heuristic, 16 
unique versus alternative, 
34-35 
optimal values, 34 
v notation, 421, 425 
optimization models, 4 


cases with local optima global, 
118 
(see also mathematical 
programs) 
OR (see operations research) 
order quantity, 4 
output variables, 9 
Oxygen System Engineering 
Design, 792-794 


P 


packing constraints, 567 
pairwise interchanges, 683 
parameters of models, 9 
indexed symbolic, 41-42 
parametric programming, 
354-362 
inadequacies of range outputs, 
354-356 
multiple objective changes, 
361-362 
multiple RHS changes, 359-360 
(see also sensitivity analysis) 
parent bounds, 668-669 
terminating by, 669 
parent of a tree node, 668 
Pareto optimal points (see 
efficient points) 
partial derivatives, 99-101 
(see also gradients) 
partial solutions, 651-652 
active, 653 
branched, 654 
completions, 652 
partitioning constraints, 568 
paths, 412 
optimal versus optimal 
subpaths, 419-421 
versus chains, 484 
penalty functions, 827 
differentiability, 829-830 
exactness, 830-831 
penalty methods, 827-833 
multiplier management, 
831-832 
sequential unconstrained 
penalty techniques, 832-833 
penalty multipliers, 827 
management, 831-33 
penalty objective function, 827 
permanently labeled node, 
440-441 


PERT (see program evaluation 
and review technique) 
Pfizer Pharmaceuticals, 795 
Phase I, 118 
artificial model, 119-121 
artificial variables, 119-120 
minimum cost flows, 497-499 
objective function, 120 
outcomes, 122-123 
starting solutions, 121-122, 
212-215 
testing infeasibility, 123-124 
(see also two-phase method) 
Phase IT, 119 
proceeding from Phase I, 122 
(see also two-phase method) 
Pi Hybrids, 39 
piecewise-linear approximation 
of separable programs, 802-803, 
860-866 
correctness, 863-866 
pivots, 230 
plant location problems (see 
facility location problems) 
points (see solutions) 
populations in genetic 
algorithms, 695 
portfolio management, 803-804 
positive definite matrices, 747 
local optimality conditions, 
748-749 
test for convex functions, 754 
positive dicycles, 458 
positive semidefinite matrices, 
TAT 
local optimality conditions, 
748-749 
test for convex functions, 754 
post-optimality analyses (see 
sensitivity analysis) 
posynomial functions, 807, 866 
posynomial geometric 
programming, 808-810, 866-874 
degrees of difficulty, 872 
dual program, 870-874 
logarithmic change of 
variables, 868 
standard form, 866 
precedence constraints, 450 
in job shop scheduling, 610-611 
predecessor, 450 
preemption in scheduling, 603 


preemptive goal programming, 
394-397 
preemptive optimization, 
384-386 
efficient points, 386-387 
prescriptive models, 12 
versus descriptive, 12-14 
President’s Library, 463 
pricing columns in revised 
simplex, 234-235, 332-334 
primal complementary slackness, 
322-323, 335-336 
primal-dual barrier search, 
291-293 
Algorithm 6C, 293 
directions, 291-292 
primals, 315 
principal minors, 747 
principal submatrices, 747 
principle of optimality, 421 
probabilistic models (see 
stochastic models) 
probability density function, 720 
process times, 602 
processor scheduling (see 
scheduling) 
program evaluation and review 
technique, 722-723 
project activity, 450 
project duration, 450 
project management, 450-458, 
Yee 
project networks, 451-452 
acyclic property, 458 
projection matrices, 258 
projection on equality 
constraints, 256-262, 267-269 
affine-scaled form, 267-268 
burden of interior point 
methods, 268-269 
obtaining feasible directions, 
257-260 
obtaining improving 
directions, 261-262 
projective transformation 
method, 251 
Purdue final exams, 58-59 
pure integer programs, 56 
(see also discrete optimization) 


Q 


QAP (see quadratic assignment 
problems) 


QP (see quadratic programming) 
quadratic assignment problems, 


576-580) 
flow-distance, 577 
formulation of TSPs, 591-592 
tractability, 584 
quadratic fit search, 734-737 
Algorithm 13C, 736 
quadratic form, 747 
Quadratic Portfolio 
Management, 803 
quadratic programming, 804-805, 
849-858 
active set methods, 852-858 
equality constrained, 850-852 
quadratic Taylor approximation 
(see second-order Taylor 
approximation) 
Quantas Airways Limited, 170 
quasi-Newton condition, 767 
satisfied by BFGS, 773 
quasi-Newton methods, 766-774 
versus gradient search, 767 
versus Newton’s method, 767 


R 


random keys, 697 
random variables, 16 
rank one matrices, 769 
ratio constraints, 138-139 
realizations of a random 
variables, 17 
reciprocals of convex/concave 
functions, 754 
rectilinear distance, 162 
linear modeling, 162-164 
versus Euclidean, 162 
recursions (see functional 
equations) 
reduced costs, 200 
as dual slacks, 33 
¢ or c; notation, 200 
cycle direction computation, 
490 
revised simplex computation, 
234-235 
reduced gradient major and 
minor iterations, 847-848 
reduced gradient methods, 
837-848 
Algorithm 14C, 846 


915 


Index 


basic, nonbasic, superbasic, 
839-840 


directions, 842-843 


reduced gradients, 841 
reflecting step in Nelder-Mead, 
779 
regression (see curve fitting) 
relaxations 
bounds, 635-637 
choosing big-Ms, 642-644 
constraint, 632-633 
Lagrangian, 647-650 
LP, 633-634 
optimal solutions, 637-639 
proving infeasibility, 634-635 
rounded solutions, 639-641 
strengthening, 642-646 
valid inequalities, 644-647 
relaxing constraints, 305 
(see also sensitivity analysis) 
release times, 602 
reoptimization (see parametric 
programming) 
reorder point, 4 
residual digraphs, 529-530 
feasible directions, 530-531 
improving directions, 532-533 
residual errors, 720 
resources 
interpretation of constraints, 
300-302 
return arcs, 515 
reverse arc, 484-486 
revised simplex search, 225-238 
Algorithm 5C, 236 
computing reduced costs, 
233-235 
pricing vector, 234 
updating basis inverse, 230-233 
use of basis inverse, 225-250 
RHS (see right-hand sides) 
right-hand sides, 47 
b or 5; notation, 184 
multiple changes, 359-360 
ranges, 341-344 
sensitivity analysis, 305-307, 
309-311, 344-346 
River Power, 651 
root node, 653 
rounded solutions 
for incumbent solutions, 
664-668 
from relaxations, 639-641 
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routing problems 
multiple routes, 592-594 
traveling salesman, 584-592 


S 


saddle points, 745 
versus local optima, 745-749 
safety stock, 5 
scalars, 79 
scaling (see affine scaling) 
schedule conflicts, 603, 611 
schedule slack, 456-457 
scheduling 
conflict constraints, 
603-605, 611 
CPM, 450-457 
disjunctive variables, 603, 611 
handling of due dates, 605 
job shop, 609-613 
objective functions, 605-609 
project, 450-457 
single processor, 602-609 
time decision variables, 603 
(see also shift scheduling 
models) 
second derivatives, 739-741 
second partial derivatives (see 
Hessian matrices) 
second-order necessary 
optimality conditions, 748 
second-order sufficient 
optimality conditions, 748-749 
second-order Taylor 
approximation, 741-743 
sensitivity analysis, 10 
adding or dropping 
constraints, 308-309 
adding or dropping variables, 
314-315 
LHS changes, 307-308 
objective coefficient rates of 
change, 313-314 
objective coefficients, 311-313 
qualitative for LPs, 304-315 
quantitative for LPs, 315-324 
relaxing versus tightening 
constraints, 305 
RHS changes, 305-307 
RHS rates of change, 309-311 
(see also parametric 
programming) 


separable functions, 801, 858 
separable programming, 801-803, 
858-866 
convex, 864-865 
piecewise-linear 
approximation, 
802-803, 860-866 
standard form, 858-859 
sequential decision making, 
458-459 
sequential unconstrained barrier 
technique, 836 
Algorithm 14B, 836 
sequential unconstrained 
min/maximization techniques, 
827 
sequential unconstrained penalty 
technique, 832-833 
Algorithm 14A, 832 
Service Desk Design, 827 
set covering constraints, 567 
set covering problems, 566-572 
set packing constraints, 567 
set packing problems, 566-569 
set partitioning constraints, 568 
set partitioning problems, 
566-569, 572-574 
with column generation, 
572-574 
setup costs (see fixed charges) 
Shell Oil Company, 553 
shift scheduling models 
air crews, 572-574 
covering constraints, 151-152 
LP models, 148-153 
Shipley Company, 406 
shortest path problems, 413 
acyclic digraph algorithm, 
449-550 
all to all, 409-413, 433-440 
Bellman-Ford algorithm, 
426-432 
Dijkstra algorithm, 440-446 
Floyd-Warshall algorithm, 
433-439 
functional equations, 421-426 
network flow formulation, 
512-513 
nonnegative costs, 440-446 
one to all, 413-414, 426-433, 
440-446 
one to one, 414-417 


solving by LP, 426 
tractability, 426 
shrinking in Nelder-Mead, 
780-781 
signomial geometric 
programming, 874 
Silva and Sons, Limited, 369 
simple lower bounds, 238 
simple upper bounds, 238 
simplex algorithm (see simplex 
search) 
simplex cycle directions, 525-526 
infeasibility with degeneracy, 
525 
simplex dictionaries, 207-209 
relationship to improving 
search, 210-211 
simplex directions, 198-200 
feasibility, 198-200 
infeasible with degeneracy, 
221-222 
minimum cost flows, 525-526 
test for improving, 200-201 
simplex search 
Algorithm 5A, 204 
basis update, 203-204 
degeneracy, 215 
directions, 198-200 
finite convergence, 224-225 
global optimality, 206-207 
lower- and upper-bounded 
form, 238-245 
revised form, 225-238 
starting basic solution, 197 
step sizes, 201-203 
two-phase Algorithm 5B, 215 
two-phase approach, , 211-224 
with dictionaries, 207-209 
with tableaux, 209-211 
(see also lower- and 
upper-bounded simplex 
search) 
(see also network simplex 
search) 
(see also revised simplex 
search) 
simplex tableaux, 209 
correspondence to improving 
search, 210-211 
simulated annealing, 690-694 
Algorithm 12E, 691 
temperatures, 691 
simulation models, 12, 17-18 


simultaneous linear equations, 
193-194 
single commodity flows, 511 
versus multicommodity, 
536-538 
single complement 
neighborhoods, 685 
single machine scheduling (see 
single processor scheduling) 
single objective optimization, 
59-60 
versus multiobjective, 63-64 
single processor scheduling, 
602-609 
(see also scheduling) 
singular matrices, 193 
relation to linear dependence, 
194 
testing by determinants, 193 
sink nodes, 477-480 
slack variables, 180 
converting main inequalities to 
nonnegativities, 180-182 
smooth functions, 99, 724 
versus nonsmooth, 724-726 
soft constraints, 391 
goal program modeling, 
391-393 
solutions, 78 
as vectors, 78-81 
encoding in genetic algorithms, 
696-697 
source nodes 
in network flows, 477-480 
in shortest paths, 413 
spanning trees, 522 
network flow bases, 522-523 
staff planning models (see shift 
scheduling models) 
stages of a dynamic program, 462 
standard forms 
affine-scaled LPs, 266-267 
convex programs, 797 
differentiable NLPs, 818 
linear assignment, 509 
linearly constrained NLP, 837 
lower- upper-bounded LPs, 
238-239 
LP duals, 288 
LPs, 180-186 
mathematical programs, 26, 
46-47 


minimum cost flows, 477, 481 
posynomial geometric 
programming, 866-868 
quadratic programs, 849 
separable programs, 858-859 
transportation problems, 504 

star * notation, 5 

start node, 451 

starting feasible solutions, 118 
basic, 197 
big-M methods, 124-127 
minimum cost flow, 497-499 
two-phase method, 118-124, 
212-214 


states of a dynamic program, 459 


static models, 153 
stationary points, 743 
necessary optimality 
conditions, 743 
of Lagrangians, 812-813 
steepest ascent/descent 
directions, 104-105, 760 
step sizes, 88-89, 737 
active set OP, 852-853 
affine scaling, 271-273 
cycle directions, 491-492 
improving search, 94-95 
lower and upper bounds, 
241-242 
LP standard form, 201-203 
minimum ratio rule, 202-203 
Nelder-Mead, 778-779 
Newton step barrier, 281-284 
reduced gradient methods, 
844-845 
zero due to degeneracy, 
221-222, 841 
stochastic models, 16-19 
versus deterministic, 18-19 
stochastic simulation, 17-18 
stopping early in branch and 
bound, 670 
error bounds, 670-671 
strong duality, 321-322, 331-335 
subpaths, 419-421 
subscripts, 40 
components of vectors, 79 
(see also indexing) 
subtour elimination constraints, 
587-589 
subtours, 587 
summation notation, 42 


Index 


SUMT (see sequential 


unconstrained min/maximization 


technique) 
superbasic variables, 839 
Superfi, 583 
superscripts for distinguishing 
vectors, 79 
supplies, 301, 478 
supply nodes (see sources) 
surplus variables (see slack 
variables) 
Swedish Steel, 135, 555 
Swift Chemical Company, 172 
switching constraints, 558, 596, 
601 
big-M constants, 642-644 
symmetric matrices, 259 
quasi-Newton requirement, 
767 
symmetric quadratic program 
form, 849 
symmetric traveling salesman 
problems (see traveling 
salesman 
problems) 
Syntex Laboratories, 878 
system boundary, 9 


T 


tabu list, 687 
tabu search, 687-690 
Algorithm 12D, 688 
tardiness, 605 
target levels (see goal levels) 
Taylor series approximations, 
741-743 
temperature in simulated 
annealing, 691 
temporarily labeled node, 
440-441 
termination in branch and 
bound, 653 
by bound, 658 
by infeasible, 658 
by parent bound, 669 
by solving, 659 
Texaco, 789 
Texas Transfer, 413 
three-point patterns (see 3-point 
patterns) 
tight constraints (see active 
constraints) 


917 


918 


Index 


tightening constraints, 305 
(see also sensitivity analysis) 
time horizons, 156 
time-expanded networks, 
517-519 
time-phased models 
balance constraints, 154-155 
LPs, 153-157 
network flows, 517-519 
time horizons, 156-157 
Tmark, 595, 645 
Top Brass Trophy, 175 
total enumeration, 628-630 
exponential growth, 629-630 
total supply = total demand, 
480-481 
total unimodularity, 501 
tours in traveling salesman 
problems, 585 
tractability, 11 
assignment and matching, 584 
constructive search, 703 
continuous versus discrete 
variables, 53-54, 144 
convex programs, 799-800 
engineering design NLPs, 792 
gain/loss flows, 543 
goal programs, 397 
improving search, 88, 109-118 
Lagrangian relaxations, 649 
linear versus nonlinear 
functions, 51 
linearly constrained NLPs, 789 
longest path problems, 458 
LP versus network flows, 
499-500 
LP versus shortest path, 426 
LPs, 176-177 
multicommodity flows, 540 
network flows versus shortest 
paths, 512 
posynomial geometric 
programs, 809-810 
quadratic programs, 805 
separable programs, 802-803 
shortest path problems, 420 
single versus multiobjective, 
63, 397 
smooth versus nonsmooth 
functions, 724-726 
unconstrained versus 
constrained NLPs, 717 


unimodal objective functions, 
109-112 
versus validity, 11 
transportation problems, 502-506 
standard form, 504 
transportation tableaux, 510-511 
transpose of a matrix, 259 
transshipment nodes, 477-481 
traveling salesman problems, 
584-592 
formulating asymmetric, 
589-591 
formulating symmetric, 
586-589, 591-592 
heuristic methods, 680-698 
quadratic assignment 
formulation, 591-592 
subtour elimination 
constraints, 588-589 
subtours, 587-589 
symmetric versus asymmetric, 
585-586 
trees, 522 
TSP (see traveling salesman 
problems) 
Tubular Products Division, 139 
twice differentiable functions, 
740 
Two Crude Petroleum, 23 
two-phase method, 118-124 
Algorithm 3B, 119 
simplex Algorithm 5B, 215 
(see also Phase I) 
(see also Phase II) 


U 


US. Air Force, 173 

U.S. Department of the Treasury, 

550 

US. Forest Service, 132 

US. Marine Corps, 504 

US. Navy, 407 

US. Postal Service, 716 

unbounded models, 37-39 
detection with improving 
search, 98-99 
detection with simplex, 219 
primal to dual relationships, 
336-338 

unconstrained global maxima 

(see unconstrained global 

optima) 


unconstrained global minima 
(see unconstrained global 
optima) 
unconstrained global optima, 112 
convex/concave objectives, 
752-753 
(see also global optima) 
unconstrained local maxima (see 
unconstrained local optima) 
unconstrained local minima (see 
unconstrained local optima) 
unconstrained local optima, 112 
first-order necessary 
conditions, 744 
second-order necessary 
conditions, 748 
second-order sufficient 
conditions, 748-749 
(see also local optima) 
unconstrained nonlinear 
programming, 715 
1-dimensional search, 726-737 
3-point patterns, 731-734 
BFGS search, 768-774 
curve fitting, 718-720 
global optimality conditions, 
752-753 
golden section search, 727-731 
gradient search, 757-761 
local optimality conditions, 
743-749 
maximum likelihood 
estimation, 721-723 
Nelder-Mead search, 774-781 
Newton’s method, 761-766 
quadratic fit search, 734-737 
quasi-Newton methods, 
766-774 
single-variable, 717 
unconstrained optima (see 
unconstrained global optima) 
undirected graph, 414 
directing, 415-416 
(see also graph) 
unimodal objective functions, 
109-112 
linear, 111-112 
single variable, 726-727 
tractability for improving 
search, 109-112 
versus convex/concave, 
756-757 
unique optimal solutions, 34-35 


United Parcel Service, 551 

unrestricted variables, 183 
converting to nonnegative, 
183-184 

URS (see unrestricted variables) 

USAF, 173 

USPS, 716 


V 


v notation for dual solutions, 
234-235 
valid inequalities, 644-646 

in branch and cut, 676-679 
validity, 11 

simulation models, 12 

versus tractability, 11 
value space, 382 
variable costs, 558 

versus fixed, 558 
variable elimination methods, 
837 
variable-type constraints, 24-25 
variables (see decision variables) 


variance of return, 803 
vectors, 79-81 
addition and subtraction, 80 
boldface lower case notation, 
79 
dimension, 79 
distinguished by superscripts, 
19 
dot product, 80 
geometric interpretation, 79 
multiplication by matrices, 185 
norm or length, 80 
scalar multiple, 80 
verbal models, 4 
vertices (see nodes) 
Victoria, Australia, 622 
Virginia Prestress, 161 


Ww 

W.R. Grace, 621 
Wagner-Whitin lot sizing, 
459-462 

warehouse location problems 


Index 919 
(see facility location problems) 
Wastewater Network Design, 599 
We Build Construction, 451 
weak duality, 289-290, 329-333 
weighted sums 

dot product notation, 80-81 

linear functions, 111 

of convex/concave functions, 

754 

of deficiencies, 392-394, 

396-397 

of objectives, 387-389 

of vectors, 194 
what ifs (see sensitivity analysis) 
WLP (see warchouse location 
problems) 
work in process, 606 
workstations, 507 


Z 
zigzagging, 761 


